But as you can see, the kernel is not only 2d, but 3d. And now the sliding, so if we


RepNet: Counting Out Time - Class Agnostic Video Repetition Counting in the Wild (Paper Explained)

have actually more than that, the sliding is not only done into the direction of height


and width, but also into the direction of depth, so that each of the frames each of


the video frames right here can incorporate information from its immediate neighbors in


case of a three by three by three filter or even more neighbors. But here we use a three


by three by three filter. Okay, so that's, that's how we obtain these embeddings right


here. And at the end, there is a dimension reduction, like a max pooling or something.


But ultimately, what you'll end up with is for each of the 64 video frames, you get one


vector. And that vector mainly describes that particular video frame, like if you consider


the green one here, but also contains information from the video frame before it and from the


video frame after it. Okay, so that's sort of and that's, that's the so the temporal


convolution is not to detect the periodic actions, because it's just one frame into


the future and into the past, it is more like what we said here, in order to give you extra


information of what's happening in a particular frame. Because especially for periodicity,


it's actually important if the arms are going up or down. Cool. So then comes the heart


of the architecture, the heart of the architecture is this temporal self similarity matrix. Now,


what does this do? This is relating the frames to each other. And important to note here,


this is just a single channel image. So there is no other ones of these for the entire video


frame sequence, you have one 64 by 64 matrix, and all the signal has to go through this


matrix, right? Everything from here is only through this matrix, there's no residual connections,


there's no skip connections. That's all that's your information bottleneck. And by having


a bottleneck like this, these, the authors here force the model to basically do a good


job at making this temporal self similarity happen if it wants to achieve a low loss.


That's what I mean by you can achieve a lot by having a smart architecture. This temporal


self similarity matrix is actually not learned, this can be computed deterministically from


the embeddings right here. So what you do is you each row here corresponds to one frame.


So you take each frame that corresponds to a row, and you calculate simply the the distance


to or the similarity with each of the other frames. So this is as you can see, it's 64


by 64. So frame i here is simply compared to each of the other frames j. And it depending


on whether that embedding is very similar to the embedding of frame j, this number is


going to be high or low. Now there's also a after that you do a softmax across here


such that this is going to be like a distribution and not just raw numbers. But that's ultimately


not that important. So what you can see is that the diagonal is very prominent. And that


makes sense because technically, the diagonal is always one, right? If for example, if you


use the inner product as a similarity, the diagonal should always be one. But here we


have the softmax, so it's not but ultimately, we can say that any frame should be very similar


to its is going to be very similar to itself. So that's why. But then you can see right


here, there's a pattern emerging. And that pattern is these diagonal lines in this direction,


as you can see. And what what does that mean? They actually have a larger version of this


down here. So what does the diagonal pattern mean? It means that so here the diagonal,


that's okay, frame i is very similar to frame i. Cool. But the other lines, what do they


mean? So if I look at frame i right here, it is also very similar to this frame j. Now


this wouldn't be further, you know, frames are similar. But the line means that if I


have if I look 10 frames later, I plus 10. That's very similar to j plus 10. And if I


now look at i plus 20 frame, that's very similar to j plus 20. So this is the this is here


why the the pattern is emerging of a line. Because if I go 10 frames into the future,


it's similar to the other one 10 frames into the future and so on. And that means the line


indicates that this entire sequence starting from i 20 frames into the future is repeated


starting from j, actually j is earlier here. So but you get the point. And if I have a


bunch of these lines, that means that this sub sequence is repeated again and again and


again throughout the video. So each of these lines is basically one repetition of the sequence


from the middle here at some other point in the video. And that's pretty fascinating.


And that's these these self similarity matrices. That's what they're sort of showing. Now they


don't use the inner product here as a self similarity metric, they actually use as you


can see right here, they use the negative squared distance, but the effect is the same.


So negative squared distance followed by a row wise softmax operation. So you could say,


Hi, we're basically done having the self similarity matrix, what we could do, let's let's say


we could train it, we don't worry about how to train it, we could simply take each row


right here. And we plot the intensities across that row. And that's maybe you know, like


something like this, and then there's the diagonals a bit higher. Okay, we could just


use a heuristic to detect these bumps here, basically calculate the length, count the


bumps and calculate the period length, right that that should be pretty easy with like


a simple heuristic. But the authors here, they want more, they want to solve more problems.


So what are some of the problems, we already saw some of the problems namely, for example,


there is the hammer throw. So the hammer throw starts out slow and gets faster and faster


and faster. And you can see this pretty clearly at the lines right here. Namely, if you go


through time, so you start off here, and you go through time, you can see that the distance


to the next line here is fairly large, but you go through time further, the distance


gets shorter, you go through time further, the distance gets even shorter. So these pattern


of lines here, that's kind of converging towards that, it indicates that this repeating action


gets faster and faster and faster. This is nice to see her at the bouncy ball example,


where you can see it starts out pretty slow, but it gets faster and faster. And here, you


if you if you have this full thing right here, that basically means all the frames are self


similar to each other, which basically means if you stop the video, right, that's if you


have 10 frames in a row, the same thing, the ball is just lying on the ground, all of these


frames will be self similar. So there's probably no bouncing happening down here, you can see


pretty well from the pattern, what happens. And here in this mixing concrete example that


we saw at the beginning, you can see that at the beginning at the end, there's this


intro and outro sequence. And only in the middle is there a repeating action. And that's


indicated by this line pattern is only at the in the middle of the videos only between


here and here. So it's, it's going to be pretty difficult to just have a heuristic that reads


out these periodic action periodicity. And in a true deep learning fashion, the the authors


here, oh, sorry, maybe you can't see that I've shifted my recording window. So maybe


sometimes something's out of frame and you have to yell at me if I do that, please. So


I hope you saw this that you have the ever the speeding up here and here, they're visible


in the pattern. And then here, you have the beginning sequence, the end sequence that


have no repeating pattern, and the repeating pattern only moves in the middle. So the authors


want to do this through, of course, a deep learning network, they want to read out the


periodicities, not through a heuristic, but using a deep network, you know, respectable,


that's at the times we live in. So what do they do? First of all, you have to see right


here, everything that happens from here, as I understand it, is per frame. So they simply


take a row of this matrix right here, like this red line, and that is independently pulled


through to the end. So there is no interaction happening anymore between the individual frame


data, the only interaction that happens is a little bit here at the temporal convolutions.


But the only real interaction between the frames is happening through the self similarity


matrix. And again, this is the information bottleneck that the authors force the information


through everything happening from here. Now, that's actually not right, there is this convolution


right here. But still, this is the information bottleneck you have to go through. So right


here, we process this image using a convolution. So this is an image, right? And we can process


it using a convolutional neural network. So what we do is we have a 64 by 64 image in


one channel, we simply up sample that not up sample, but we expand the channels to 32


channels. Now, as I said, it's pretty easy to think we can just go to the end here, use


a convnet to produce our final 512 by so 512 embeddings we have here, again, 64 by 64,


that we then use to predict the final result. But the authors here do something different,


they do transformer layers in the middle, but only per frame. So what does it mean?


So here, you up sample to 32 channels. And then that means that one of these blocks right


here, one of these blocks corresponds to one row in the self similarity matrix, which corresponds


to one frame. And from now on, so from now on, I want to say what I said before, from


now on, it's all just this one block, they are independent of each other. Okay, so you


take this one block, and you feed it through a transformer to achieve at your final embedding


of 512. And it's probably best if we read what they say about it. Okay, so if we're


given this self similarity matrices, matrix, they consist of row, each row is the per frame


self similarity representation, and generates two outputs, the per frame, period length


estimation and the per frame binary periodicity classification. Note that both L and P are


vectors and their elements are per frame predictions. Okay. The architecture of the period predictor


module can be viewed in figure two, note that the predictors share a common architecture


and waits until the last classification phase. The shared processing pipeline starts with


32 2d convolutional filters of size three by three, followed by a transformer layer,


which uses a multi headed attention with trainable positional embeddings in the form of a 64


length variable that is learned by training. Okay, it's, I guess the transformers learned


by training, and the positional embeddings are also learned by training. That's fairly


common. We use four heads with 512 dimension in the transformer. By the way, if you don't


know what a transformer is, watch the video on attention is all you need. I made one it's


very popular. Yeah. So with each head being 128 dimensions in size. After the shared pipeline,


we have two classifiers, period length classifier and periodicity classifier, tau. Sorry, this


is fine. This is tau. Each of them consists of two fully connected layers of size 512.


So I guess the the pipeline here is pretty simple. The question could be why do they


use a transformer and not simply another convolutional network. So here they up sample the image


as we saw into 32 channels, and then they simply want to take one of these one of these


blocks here. And that corresponds a little bit. So we have for one frame, right? What


does it mean we have basically, we have 64 by 32 things. So the 64 things, it's this


one frames temporal connection to each other frame given that you know, comes from this


self similarity matrix. So it kind of relates this frame that we're considering to each


of the other frames. And each of this, each of these entries is a 32 size vector. This


is sort of a this is you can consider it like a sequence of 64 things 64 embeddings. So


to use a transformer here is pretty natural. If you think of this as like a sequence transformation


task, I would guess. So the transformer can if there are these peaks right here, like


we saw right here, the transformer can make very good sense of that because of course,


the attention mechanism from a one peak, it can attend to all the other peaks and can


sort of relate the different peaks to each other and then determine the periodicity length


whereas with a convolutional network, I guess that's going to be a lot harder because of


the sort of invariants built into the convolution. I'm not sure maybe they also it just worked


better. But that's how I think about it. It's that for a given frame, you basically have


a sequence classification or a set classification task. And the attention mechanism allows you


to in one single step connect each peak with each other peak or each information with each


other information in this sequence. Alright, so at the end, you have just fully connected


layers again, only on a per frame basis. And that will give you the output. And again,


you compare this to the label and you back prop through everything, everything here is


differentiable. So all of this is trained to achieve minimum possible loss. And because


you train everything to achieve minimal possible loss, you make this encoder right here, which


is the crucial part because the encoder is must give you good embeddings, which must


give you a sensible self similarity matrix, right? You train the encoder to encode things


that are relevant for the task. And that's what makes the whole thing work. Okay. So


we've gone through the architecture. Now, the problem right here is the the data set.


So they also go into how they do inference, they can actually do a bunch of things like


play the video at different speeds and then look at what each of the predictions. So if


a double speed, it predicts half the period length, then you can be more sure and so on.


So that's pretty cool. But they go into another point right here, and that's the data set.


So they produce this countix data set. But also, on the other hand, which is something


I also find very cool is they produce a synthetic data set. So here they say, we train with


synthetic repetitions. And that can be sort of, I didn't know what to think of it at first,


I was just like, huh, but then it's pretty cool. So if you have a video with these, these


are the frames of the video, right, so the video goes in this temporal direction, what


you can do is simply go here, go through these frames, and just repeat these frames and repeat


them and repeat them. And at the end, you have these frames, right. And then you have


a data set. And if you if you assume that most videos do not naturally contain repeating


actions, right, most videos are just videos, they're not videos of something repeating,


then you can safely assume that these parts here are non repeating. So and these parts


here are repeating, this is one of the labels that you need, right? The problem with synthetic


data set is always to have the labels. And also, you know how many there are, because


you can simply count the number of times that you go through it, you can even make it faster,


slower, and so on. So this synthetic approach is pretty cool. And especially the bottom


right here, because this might be kind of hacky. Because each time, each time you jump


from the end of one of those arrows to the beginning, right, you have kind of a hack


in the in the video, because, you know, it's not continuous. So what you can do, and this


is the the bottom here, you can do this reversal technique where you go to the end, and then


you play the frames backwards, and then you play the frames forwards again, backwards


again, forwards again, and then you go out here. And that gives you one continuous motion,


right? If someone if it's simply a video of someone lifting their hand, like it starts


out down here, and it goes here, and it goes here. And then if you do this technique, it


would go down again, down again, up again, up again, and so on. So that's, you know,


I think it's a fairly smart technique, honestly. Now they tried this, and it doesn't work super


well. So what they also have to do is they have to do manual camera motion augmentation.


So that's camera motion augmentation, it basically means that if you just do a repeating action,


like this, it's sort of, I guess it's too monotonic, it doesn't really cover real videos


with repeating actions. So what they do is they kind of simulate a moving camera. And


you simulate that much like you would do image augmentation. So you can rotate the camera


over time, you can translate it, you can scale it differently. And through if you do that


throughout the video, and you change it around how the camera moves, then that appears to


work fairly well. So if they now compare this and their data set, they perform pretty well.


So in their data set, they take this kinetics data set, and they crowdsource the label and


the tasks in the data set, they're pretty diverse, as you can see right here. So you


have sports like rope training, mountain climbers, but you have also things like playing ukulele,


exercising arms, slicing an onion, and so on. And you can see that the repetition count


is fairly diverse as well. So from one or two repetitions per video, it goes to 50 or


so. And the period length is also between one and five seconds, though, as you as I


already said, you don't have to, you don't have to count on that because you can always


play the video slower or faster and then determine other periodicities. So in their experiment,


first of all, they perform pretty well. And they show that if they train on their data


set, and on the synthetic data set, they perform better than if they just train on the synthetic


or they just train on their data set. They also show pretty clearly that the addition


of this temporal self similarity matrix helps tremendously, you can see right here in each


of these boxes is the comparison. And this OBO, I think is the off by one error. So it


kind of forgives you if you're off by one count, but otherwise, you get a zero if you're


wrong. And you can see that the self similarity matrix helps tremendously. They also compare


with some other architectural choices instead of the transformer, I guess, yeah, so I guess


they just take it because it performs pretty well. And they do a lot of lot of ablations.


But what I particularly appreciate is that they do something like this. So what they


do at the end, when once they've trained the architectures, they do a 1d PCA projection


of the encoder features over time. Now, the encoder features, they were 512 dimensional,


right? This is the thing before it goes into the self similarity matrix. So those we said


the encoder is the crucial part here because it needs to take the video and encode things


that make them accessible to calculating the self similarity. Now they do a 1d PCA. So


a projection into one dimension of these features. And you can already see at this one dimensional


projection that the periodicity here is clearly clearly visible, namely, for example, right


here. Every time up here is when the legs are up. And every time down here is when the


legs are down right here. So that is very, very impressive. And that kind of that really


shows that the model is doing what you claim what you claim that it's doing. Like, I'm


almost more interested in experiments like this than in and in these numbers right here,


because the numbers could always be because you've just thrown more stuff at it, right?


So they go over a bunch of possible applications of their model. So first of all, you can do


something like as we can see, repetition counting from videos, you can do periodicity detection,


those were the things that the model is trained to do. But there's also a bunch of things


that the model can now implicitly do, namely, something like change inspection, where they


say, Look, if someone's chopping this pineapple right here, then at the end of each of the


repetitions, there is something that changed namely the number of slices of pineapple is


it red? Is it? I can't. I think it's pineapple. Okay, so the number of slices or pieces right


here changes. So in essence, this could be the base for another model estimating whatever


changed or training to recognize numbers of pieces and so on. Also, you can detect the


speed. So the speed of a repeating action, if you perform something slow or fast, this


model can implicitly do it. And this they call cross period retrieval. So if you know


when the repetitions are, you know that, okay, maybe the first frame, so always on the upswing


right here, these should all these should all be fairly similar visually, right? As


with respect to the repeating action. So you can see that even though this whenever the


kid in the swing here is close, it looks fairly different in a purely visual sense in a pixel


sense, but it is at the same point in the repeating action. And that's, you know, that's


that's pretty cool. So you can technically retrieve related things, even though they


visually they don't look similar that much. Yeah, that that's the, the kind of applications


here are probably many, many fold. And I also think that so in this measure of intelligence


paper by Francois Chollet, he basically claims that this is one of the innate abilities of


humans. They can count, you know, they can count things, this is something you're basically


born with. And maybe this thing right here will become sort of a staple staple component


for many other things that we build AI on, I would not be surprised, but maybe it will


just fade into history. I think it's pretty cool project, especially, you know, the the


architectural choice here to pull everything through this self similarity matrix. And the,


you know, just just looking at this matrix already makes you kind of know that this thing


works. Alright, this was it from me. Let me know in the comments what you think about


the paper, check out the website, the website has a lot of video demo examples of what they're


[ML News] GPT-3 learns to edit | Google Pathways | Make-A-Scene | CLIP meets GamePhysics | DouBlind

GPT three learns to edit text text to image generators achieve new heights. And Google


finally introduces their pathway system. Welcome to ML news.


Quick work from our sponsor weights and biases. If you don't know weights and biases, you should


definitely check them out. They are the best when it comes to MLOps. It's the entire package,


they will automatically track your experiments, send everything to the cloud track your models,


your outputs, you can even give them your data sets, they tune your hyper parameters,


they make everything shareable with your team and with the wider world is really cool. Today,


I want to highlight this report that I found by Scott Condren. So it's a little bit of a showcase


what you can do in a one to be report. And what he's showing here is sort of a before picture


where people took screenshots of TensorBoard log plots or even matplotlib plots. Now he made it a


bit pixel ish on purpose, but I've definitely seen things like this in papers crazy, but no more with


weights and biases reports, you can share your research with the highest quality available. So


let's say you've tracked a bunch of experiments and you want to present the best ones people can


check them out interactively you see right here, I can go I can zoom in, I can click on a run,


I can inspect that run in detail, like what were its hyper parameters, how much CPU and RAM did it


use? What was the console log output of that run? Everything is observable. But not only that, let's


say I want to communicate how different hyper parameters affect the final objective. Well,


the best way to do this is a plot like this, this shows me all the runs in different hyper


parameter configurations on each of these axes and where they end up in the final loss. Again,


this is fully interactive, and you as the writer of the report can place it wherever you want. But


it's not only about experiments, reports can also include one to be tables and tables are really


cool. Tables are like an Excel sheet on steroids. And again, this is fully interactive, I can


inspect any cell here. So you can even interactively modify these tables. So I've actually introduced


a column in this other person's report that shows me whenever the ground truth label doesn't agree


with the model, and I'm able to sort by this and explore wherever the model makes mistakes. This


is really neat because it decouples who runs the experiments and the evaluations from who does the


analysis on the data. So this is just a small set of features that you can do in reports and they


work especially well within teams or collaborators worldwide. Again, I invite you to check out


Weights and Biases. They've been really great sponsor go to wonder be.me slash Yannick to


let them know I sent you and now let's get into the video. All right, hello, everyone,


it's Monday and a new episode of ml news. wide angle camera really nice. You see more of me.


I don't know if that's a good thing. GPT three gains new editing capabilities. So if you don't


know GPT three is a language model by open AI. It's been available through their API,


you can go to it, you can ask it to produce text and code. And now they've added a new feature


that allows you to actually edit text and code. They have a bunch of demos right here where they


write a piece of code and then ask the model to change it in some way, for example, to make the


fibonacci computation use memorization, and then interestingly to translate it from Python to


JavaScript, which is quite impressive. Now, as I said, this doesn't only work for code, it also


works for text. And I just thought we'd give it a try. Alright, so I'm here in the open AI API.


And what I can do is I want to go and select the codex edit model, you can see right here,


you have different modes, there's the complete mode, which gives you the traditional models,


there is the insert mode, which gives you the new insert capabilities, and the edit mode again,


with the edit capabilities. Alright, so let's come up with a simple function.


Cool. So now that I have this, I can instruct the model to do all kinds of things. So here in the


instructions, I'll say, make a doc string. This is a doc. Well, okay, we might have been oversold


a little bit. Let's try. Let's try a bit more generate this functions doc string, this function


squares its argument. Excellent. Nice. Add parameter in for nation to the doc string.


Nice. All right, we're getting somewhere. Add type hints.


Look at that. Here, there's a button use as input. I'm dumb. Alright, now let's try this


translate to Java script. Boom, doc strings been translated functions been translated. Excellent.


Yeah, I can definitely see how this is powerful. Let's try another one.


Okay, this is a short recursive implementation of a depth first tree search. Now it does have


some tricky bits. For example, we're using implicit return value of none in Python, and we're never


telling it what the type of node is, we just make it have some properties that are implicitly


assumed. So let's see if it gets what this is generate an accurate doc string.


Add a doc string to the DFS function. Whoa, whoa, whoa, whoa, whoa, whoa, whoa, whoa, whoa, whoa,


whoa, nice. Okay, let's see if it gets the types add type hints. Whoo. Okay, very cool. All right,


now the super challenge translate DFS from a recursive to an iterative algorithm.


Yep. That's it. Very, very nice. Okay, there's one thing that I always wanted to do, but it's not an


edit mode. Okay, checks if the program holds return not halts program plus. I guess the ancient


computer scientists would be happy with that answer. Cool. Remember the open AI API after a


long time of being closed beta waiting list whatnot is now available for access to everyone.


So if you want, you can go play with this stuff. There's a new paper out of meta called make a


scene scene based text to image generation with human priors. Now this pushes the state of the art


in image generation from text. So here are a bunch of examples. For example, the painting of blue


elephant or a teddy bear with a blue scarves and eyes tilted to its left, like these are really


accurate and really high quality productions. Now there is a bit of a difference between something


like this and Dali or glide, which is that this takes a number of auxiliary inputs. For example,


it can take a segmentation map, which you can see here in the middle of the generated images,


it can also take reference images from which it will copy over the visual tokens. So there's more


information provided to the model. But in return, you get a lot better quality output. Now one cool


output of this is the illustration of a story that the author has made and put on YouTube. So the


story is called the little red boat. And all the images are illustrated by this model. The little


red boat woke up near the shore one day, where are all his friends, he couldn't say he decided to set


sail to the open sea to find out where everyone could be. So the story in itself is pretty neat.


And I think it gives a nice outlook on the near future we can expect out of these models. Like


since I've made my music video, we've come such a long way. And that's not too far back. So the


progress in this field is absolutely astounding. So finally, the pathways paper is out Google has


talked about this in a blog post before by Jeff Dean, and we've reported on that. But as of that


point, it wasn't really clear what pathways was, I was more under the impression that it is kind of


a new model architecture where Google wants to build like these giant models that have multitask


components, and you would only update them sparsely and so on. However, this paper right here


describes more of like an infrastructure side of things. Now, I don't know, but given that it's


called the same and it is is come out of the same company. I'm pretty sure that you know, this is


actually what they meant. Hi, this is Janek during editing. And Jeff Dean has just posted a tweet


that says this paper is about the pathway system that is designed to support the broader pathways


vision of creating large scale multitask multiple models with flexible support, the yada yada yada.


So it appears that even though the paper is called exactly the same as the vision, the two are


separate things and one is in service of the other back to the video. So what is pathways? The best


way I can describe it is something like MapReduce for machine learning. So imagine you have all


these data centers and you have all these accelerators around and some are connected


with super fast, infinite band and some are connected with a network latency, what pathways


allows you to do is to super efficiently distribute your computation across any number of devices and


in a heterogeneous way. So while we've become pretty good at something like single instruction,


multiple data computation, where we simply distribute data to different accelerators and


then run the exact same thing on all of them until we synchronize them again, heterogeneous computation


is a little bit more tricky. So if I want something to happen on one part of the data,


but then something else on a different part, like that's a problem, especially if the things take


different amounts of time, then one is idling and so on pathways is essentially a very, very smart


compiler and scheduler to distribute computation across whatever. Now I'm not knowledgeable enough


in hardware and the interconnect between how you trace your functions in your ML programs,


how the XLA compiler then figures out how long everything takes and then asynchronously schedules


everything in parallel to absolutely optimize your throughput. But this is essentially what's


happening right here, I invite you to read the pathways paper because it is very detailed and


gives you good overview over what's to come in the future. Now presumably Google is going to


deploy these things in their own data centers, which either means that you can expect faster


ML workflows on GCP, maybe the prices will come down, or maybe they'll just make more profit,


anything could happen. Dope blind is a social peer review platform. This is a website where anyone


can go and review any paper. So this is an open platform, you can make an account, you can search


for a paper, you can see what reviews already exist, and you can post your own reviews. And


this can happen in a personalized or in an anonymous fashion. Now they've already indexed as


far as I can see most of the machine learning papers, but most of them obviously don't have


any reviews yet. So I've searched for myself right here. And I agree with the zero out of five star


rating, although I think they should have like one like one is generous. But there you see the


problems with these types of platforms. Now, while I definitely agree that something like this would


be super valuable with all the problems that come along, you know, anyone can come here and post a


review and have bad intentions and smear other people's work and blah, blah, blah. But with all


of that, I still think it's a valuable addition. However, this only works if really the whole


community decides to make this the hub of things. And I just don't see that happening in the near


future anytime soon. Wait, that's a tautology the near future anytime soon, like that's the same.


All right, so I'm definitely excited to see what happens with these platforms. This is not the only


one, but it seems pretty cool. I have not yet seen any incentive here to cash in somehow on this,


which makes me a bit more hopeful for this one. But what I'd really like to see is this being


connected to something like archive directly, so that I don't have to go to this website to get my


reviews, but just the reviews somehow get aggregated from the whole internet to this platform. So when


I write something about the paper on Twitter, then it might be aggregated here too. And therefore,


you don't force the people onto a platform, but you simply grab what's out there about particular


papers. Now we've seen previously that something like Zeta Alpha tries to do this automatically.


But there again, that's a different business model. So we'll see what happens in the future.


I can't tell but I do welcome good intended efforts to revamp the peer review system.


This is an interesting paper clip meets a game physics. So this is a pretty simple method to use


clip to find bugs in video games. So people often upload buggy footage of video games to Reddit. And


I'm sorry that that is that is a bit like, what did you do to that horse. So video game developers


might want to structurally search through all of these videos that are played and uploaded from


people who find these types of bugs. And this is exactly what this paper does. So so they take all


of these videos, they index them using clip, and then you're able to search for them. For example,


if you search for a person flying in the air in the Grand Theft Auto five database, you'll get all


kinds of buggy clips of things that maybe should or maybe shouldn't be happening. Now, this is a


great help probably to game developers, but it does have a downside. Namely, you can only search


for the bugs that you know, exist. So this was actually a legitimate person flying in the air,


like that, I'm pretty sure that's what should happen. But let's say a user comes to you and says,


Well, all of a sudden, my character was stuck in the air or stuck in a tree or stuck in a wall,


what you could do is you could turn on the search engine. And you could search through all of the


footage of all of the people who played this game, whether or not something like this was happening


somewhere else. Now, the usefulness of this obviously goes beyond video games, you could


search any type of image or video footage through that there are some shortcomings. As I said, you


can only search for things that you know. And also right now, this is simply implemented as taking a


bunch of frames and then running them through clip and searching across them. So you're not able to


necessarily search anything that happens in a temporal fashion. In the video, there's not a


true video search, it's more like a frame search. That all being said, pretty cool project, the data


set is released. So you can try it out for yourself. Another paper that has caught my attention is


autoregressive image generation using residual quantization by cacao brain and pos tech. This is


another paper that pushes the state of the art in image generation from text. So the samples you see


here are pretty neat. And they can be generated not only from text, but also conditionally,


for example, the top two pictures are conditioned on image net classes, the bottom two pictures are


produced from a text prompt. And the core of this paper revolves around a technique called


residual quantization. Now, usually, if you do vector quantization, what you want to do is you


want to run your image through some sort of a downsampler, some sort of a feature extractor,


like a convnet, or a transformer. And then at the end of that, you quantize it into individual


chunks, individual visual tokens. What this model does is as it down samples the image in the feature


extractor, it quantizes at each stage, and then it remembers the residual of what it quantized. So it


will end up with a multi scale representation essentially of visual token plus whatever is


needed to reconstruct the finer grained stage that came before it. So this can retain potentially a


lot more information about the fine grained structure of the image and enables these really


high quality productions. Now what's also cool is that the models are available specifically,


there is a 3.9 billion parameter model available just for you to download. Now,


how you're going to run it is a different question, but it is available.


Alright, let's get into some helpful things for this week. Stumpy is a powerful and scalable


library for time series data mining. Fast Tree Shep is a package that provides algorithm for


explainability in tree based algorithms, meaning random forest, Xgboost, light GBM and so on. Yes,


there exists something else than deep learning. Imagine that jacks ton is a collection of 100


jacks exercises. If you've ever wanted to learn jacks, this might be the place. nov grid is a


variant of mini grid, which allows you to change underlying world dynamics. For example, right here,


the fact that the yellow key opens the door is exchanged at test time with the fact that the


blue key opens the door. The challenge for the agencies obviously to adjust to these new facts


at inference time, which is really hard if you've never trained on them. icec gym is a part of


Nvidia's Omniverse project. This is an engine to run physics simulations for the purposes of things


like reinforcement learning, population based learning and so on. The main focus here is scale,


you can run 1000s of these experiments in parallel if you have an Nvidia GPU. But still,


for the fact that these are physically accurate simulations, it's pretty cool. On GitHub,


they also have a repository with a bunch of benchmark environments for icec gym, everything's


available to download, check it out. And this was already it for ML news this week. It's been a bit


of a slow week, but I hope you still had fun. If you like slow weeks, please subscribe. One


Lemmatizing - Natural Language Processing With Python and NLTK p.8

Hello everybody and welcome to part 8 of our Python with NLTK for Natural Language Processing


tutorial video.


In this video we're going to be talking about lemmatizing.


So what is lemmatizing?


So lemmatizing is a very similar operation to stemming, only the end result is a real


word.


Okay, so the end result will be generally your root stem is going to be some form of


synonym or it might be the same word but some form of synonym to the original word.


So you might end up with a very different word but it will be a similar word with the


same meaning.


So for example let's go ahead and go from NLTK.stem import wordnet lemmatizer.


Now we'll say lemmatizer equals wordnet lemmatizer and let's just print some examples really


quick.


So we're going to do print lemmatizer.lemmatize and first let's lemmatize cats.


Cat gets lemmatized to cat.


Now let's do a few more.


Cats will do cat-tie, geese, rocks, python.


Let's try this again.


Okay so cat becomes cat, cat-tie becomes cactus, geese becomes goose, rocks become rock and


then just a singular python becomes python.


Okay so that's cool.


Now what about what if we did let's lemmatize better.


Let's just try this first see what it comes up with.


Better, but what if we said better and then I think it's POS equals and then better.


This is an adjective.


Let's try that.


Okay so it actually comes back with a completely different word which is good.


So then let's say best and then just to save time since every time we load this it run.


Run can actually be a noun like test run and then let's do run verb.


So we'll get a few examples here.


So best goes to best, run goes to run, run is a verb still goes to run.


So at least with best, best was a decent example of a word that becomes something totally different.


I'm sorry not best, better.


Better is the one that became a different one.


So by default some people get confused with lemmatizing because they'll throw run like


they'll do something like this.


So for example better as an adjective came back to good but what if we did what if we


got rid of POS is a.


Wait a moment and we get back a return of better.


Okay that's not what we what we expected now that we know that that's not what we're supposed


to expect.


So the default parameter for lemmatize is going to be POS equals noun.


So if you have something that is not a noun you have to pass through the part of speech


tag.


So keep that in mind when you lemmatize things but anyway lemmatizing is going to be better


in my opinion than stemming because it's going to give you an actual word a word with a meaning


and it will actually group a lot of words together and it does a better job I think


than stemming does but stemming is still useful and it's something if you're going to be intent


natural language processing you probably should know what stemming is.


No regrets teaching it but I do think lemmatizing is going to be so much more powerful.


So anyways that's just a real quick example of lemmatizing you'll find that you'll use


lemmas way more often I think.


So if you have any questions or comments anything confusing because I know these three lines


Introduction - Go Lang Practical Programming Tutorial p.1

What's going on everybody and welcome to a Go programming tutorial series.


This tutorial series is going to be practical and applied but also over the basics of Go.


The reason why I want to do it that way is because the Go programming language, I mean


all programming languages are like this, but the Go programming language because it's kind


of a low level programming language, it's kind of the case that you could go through


a basics tutorial for something like Go and feel like you kind of understood what was


going on and then you actually go to try to apply it to a project or something and you're


like I have no idea how to do this.


So my hope is that each time we learn something we're going to apply it to an overarching


objective and project and my hope is that that'll make learning Go much much simpler


but still even at the end of that the best way to learn something is to keep trying to


use it to solve various types of tasks so some of that's actually still going to be


on you but hopefully I can make some of that initial process easier.


So first of all why Go?


Why would someone like me be interested in Go?


Why might you be interested in Go?


Basically Go is built from the ground up, it was built by Google, it was built to solve


Google types of problems right so basically Go is for scale.


This would be like a systems language, web development language, anything you want to


scale, anything you want to be super efficient but you might not want to write in C++.


So for example the reason I'm interested in Go is mainly for web development so a website


like pythonprogramming.net for example it's built in Python using Flask and it's not the


slowest website in the world but it's also not the fastest website in the world, we could


make some serious speed gains there and in web development the difference of 100 milliseconds


is a major difference especially when you scale that out to millions of users so the


speed doesn't just affect the user experience but it also affects your bottom line, how


much are you paying for processing costs basically right and hosting time and all of that so


anywhere you can make gains in web development it's usually a very good thing and then the


bigger the scale the better those gains are for you.


Also in terms of just anything where you need it to be super fast or super efficient Go


is pretty much C++ speeds so anywhere where that's a good thing to you maybe you're in


high frequency trading or something like that where you really want to be as quick as possible


but maybe you don't want to write something as tedious as C.


So Go is going to be static typed like C++ unlike Python so every time like in Python


every time the interpreter hits a variable it has to kind of figure out what is that


variable and that variable could change types and stuff like that and by that I mean like


ints and floats and strings stuff like that whereas in Go that's not the case so it doesn't


need to know or it basically doesn't need to figure those things out for you basically


you are going to tell it what type is certain variables and all that kind of stuff so that's


going to give you some speed also it was just built from the ground up for concurrency whereas


Python was kind of built from the ground up to be a teaching language right and it just


so happens that it's so fun and so easy to program with that it got really really popular


but that's what Python was made for so Python's got the global interpreter lock kind of restricting


it to one thread yes there are ways around that but that's like band aids over a bigger


issue whereas Go is just concurrent right out of the box so stuff like that is what's


going to make Go faster but Go is much more pleasant to write than something like C++


anyways with that let's go ahead and get into it I think you'll learn a lot more about whether


or not you like Go by actually using Go and seeing what you think from there so to get


Go it's just go to golang.org head to the downloads and download whatever fits your


needs so if you're on Windows it's just a simple installer on Mac I have no idea it's


package I don't know how installing that's going to go for you I don't use Mac and then


on Linux it's a tarball download that install that the only thing I'll add is on Linux you'll


want to go ahead and export that path permanently if you don't know how to do that I'll put


a link in the description to the text base right up in this tutorial and you can head


there for instructions on how to set it up on Linux a little more easily other than that


you're going to need some sort of editor I'm going to be using Sublime Text people like


to make big deals about editors you can use whatever you want you can use Notepad for


that matter it doesn't really matter Sublime Text is a great editor though so you can use


that if you want that's what I'm going to be using and let's go ahead and write our


very first Go program so I'm going to go ahead and move this out of the way and I've just


got a simple directory called GoTuts I'm going to go ahead and just make a new file and I'm


just going to call it GoTut.go we're going to open that up and we're going to get ready


so the first thing in every Go program it's going to be a part of a package and for the


most part you're going to find yourself pretty much writing package main every time until


you start writing bigger things so package main and then you're going to have your imports


for now we're just going to import FMT and all your imports go in double quotes and something


to get used to especially if you're coming from Python which I assume many people viewing


this are from my channel mainly watch Python you have to use double quotes also you're


putting quotes around your imports but get rid of single quotes you're going to be done


using them with with Go also if I slip up and call Go Python and all that kind of stuff


I'm sorry it's going to happen I'm also probably going to type like print a few times I'm also


going to make quite a few pun puns about Go so get used to it so once you've done your


imports we're just going to import FMT which is like format basically it's going to be


how we can print something out basically we're ready to build out our functions so to actually


write a function you're going to say func that just starts your function you've got


your function name you've got parameters and then the function comes after that again it's


not Python unfortunately we've got braces so like many other languages so and then in


here you write your code we're just going to have some simple code it's just going to


be FMT dot capital P print line and then we're just going to say again in double quotes don't


use single quotes anymore welcome to Go awesome so we can go ahead and save that and let's


run this program real quick so we're going to come over here I'm going to open up the


command line and to run it you just do Go run and then whatever the file name is so


for me it's GoTut.Go hit enter and boom welcome to Go so that's going to go ahead and compile


and run it for you at the same time so so that's that not too much I really want to


cover here until we get to the next tutorial like I said there's just like all the little


things that you'll have to get used to also you might be wondering why did the main function


even run like we didn't ask it to run great question so we'll be talking about stuff like


that in the next tutorial also be explaining a lot of little smaller things here that you


might not have noticed if you have questions comments concerns Go puns whatever feel free


Noether Networks: Meta-Learning Useful Conserved Quantities (w/ the authors)

But the intuition is that knowing these five conserved quantities is going to tell me a bit


about what my prediction should be. And so it's kind of free information that I get to know.


Hello there, today we'll look at Neuternetworks, meta-learning useful conserved quantities by


Ferran Alet and Dylan Doblar and others. This is another one of the with the authors installations,


videos, whatever, where I just discussed the paper briefly right now. And then we'll jump into an


interview with one of the first authors with Ferran. And we'll go through the paper together.


And I think Ferran can explain this so much better than I can. And I'm also able to ask some of my


dumb questions. So this was a lot of fun. And I definitely invite you to stick around if you


already know a little bit what the paper is about, feel free to skip ahead. If you don't know what


the paper is about, the paper essentially deals with neural networks that predict dynamical systems.


And in these dynamical systems, very often there are these conserved quantities that are part of


it. For example, in a physical system, energy is conserved, momentum is conserved, and things like


this. And under this constraint, you can build in this constraint into the predictive neural network,


so that the neural network does a better job. And they build these neural networks in order to


dynamically learn these conserved quantities, and then adjust at runtime during forward propagation,


tailor the loss to conserve these quantities. And I think that's really cool. It's different.


And yeah, that's what I like about it. So pretty brief introduction, this paper, obviously is named


after Noether's theorem, which essentially, they say here loosely states the following.


For every continuous symmetry property of a dynamical system, there is a corresponding


quantity whose value is conserved in time. For example, they say a system of planets interacting


via gravity, the system is translation invariant in all three cardinal directions. Noether's theorem


asserts that there must be a conserved quantity for each of these symmetries, in this case,


linear momentum is conserved. So the symmetry in space as translations is accompanied by a


conserved quantity, which is linear momentum. Now, we don't always obviously know these quantities.


And they're not always super explicit. And they're not always exact. So what we are going to be


dealing with here is predictions of dynamical systems. And the example here is the prediction


of a video of like a physical interaction. So this is a thing here on an inclined plane,


it sort of slides down, and then collides with this other thing right here. And the goal is to


predict the next frames of this video. Now, we could just build a neural network to just to


predict these things, frame by frame by frame. And that would go certainly well, if we had a lot of


data. However, if we don't have a lot of data, what we need to do is we need to build in inductive


biases. And the inductive biases, what people usually do is they build in these symmetries


directly, for example, they build in the physical laws, they know how the world works. And they say,


you know, whether I translate it to the left or to the right, it doesn't really matter, and so on.


But building in these symmetries, and I think we know this from geometric, deep learning,


building in these symmetries is very powerful, but it can also be cumbersome, because you have to


define them beforehand. This paper goes ahead and says, you know, what's really what's a lot easier


than building in symmetries directly, is building in a constraint to conserve a given quantity. And


that is a lot easier. And there's a potential that you can actually learn it from data. And with


Nutter's theorem, we know that the two things are equivalent. So if a system conserves a quantity,


it essentially encodes a symmetry in the system. So what do we do? This is the very high level


overview over these networks, we take to this entire thing here is one forward propagation,


we take the original frame, we put it through a forward predicting neural network, which is this


f theta right here, this is a network that simply forward predicts frames, as we I said initially.


So we forward predict, forward predict, forward predict, this gives us an initial set of, of


outputs right here, these x tilde. Now, these are going to be pretty, pretty bad, not pretty bad.


But if we don't have a lot of data to learn from these, we don't expect them to be particularly


good. And that's the regime we are here. What we do then is we're trying to adjust this f thing


right here in the moment. So during the forward propagation, we're going to update our predicting


neural network by this neutral. So we're going to do an update, a temporary update to the weights


of the f network. And we're going to do this into direction of this neutral. So you can see here,


we have these networks g lying around, then g is always the same network. So what we're going to


do is we're going to feed each frame that we predicted through G, and G always being the same


network, it will output the same thing. And now obviously, if you know, given given that how I


made this introduction, you might already have guessed that g is the part that predicts the


quantity to be preserved. So what we want to do is we want to put all these things through G.


And then we want to these these will give us a bunch of outputs, right g here and here and here


and here will output some things and the things can either be a number or an entire vector,


right an embedding vector. So essentially G takes this thing right here actually takes two


consecutive frames and embeds it into some space. And now, ideally, all these G's would output the


same thing, which would mean that we have conserved some quantity and therefore encoded some symmetry.


However, initially, these G's are not going to output the same thing. So we are going to attempt


to change the f function such that the G's output more the same thing. There is a loss involved


right here. This is the neutral loss, they call it and it is defined down here. So you can see,


all this really is is it's either defined in one of two ways. Either you take the difference between


the g function of the initial frame and the frame at time point t, or you calculate the difference,


or you calculate the difference between consecutive frames. In either way, since you sum


across all the frames, this means that all the outputs of the G network will should approximately


be the same. Now, what do you do with this information? Again, we're still we're still


during one forward propagation. So what do you do with this information, you calculate this


neutral loss, which is one we just described, and then sorry for skipping around so much,


you're going to do one update step. So these are the parameters of the f network,


we're going to do one update step into the direction of the gradient. And it's the direction


of the gradient with respect to the parameters of the f network. So this is the forward predicting


network. So essentially, we are saying, how do I need to update my forward predicting network,


such that, right, such that the frames that it outputs the frames that it predicts in the future,


make it such that the g functions of all of these frames are more similar to each other or more


similar to the g function of that first frame. So we're going to in time update the f function


right here. And after that, we're going to forward forward propagate again, with this new f function


and thereby obtain our final prediction. This is one, this is like an inner optimization that we do


during forward propagation. I find this to be pretty cool. Now they just do they just do one


gradient step, obviously. Otherwise, you know, you could do a lot of things and you could like


program in Adam and Ada grad, not only one like gradient step, which is one SGD step, essentially.


But even with one step, that is good enough. So again, they here is the entire training procedure


in an algorithm, you can see that. Let's start down here. They start with randomly initialized


weights. These weights here are for the g network, these weights are for the f network. They sample


batches for each batch, they predict the sequence. Now the sequence prediction is this entire thing


we just looked at. So the sequence prediction is, I'm going to start at the initial frames,


I'm going to use the the f the original f the one I currently have unconditional, let's say to


forward predict all of the frames once, then I'm going to put all of these predictions here into


this neutral loss, I'm going to calculate the gradient, how do I need to update this f for this


particular data point to make the g functions output, the more similar things, I'm going to


take new parameters, again, these are just temporary parameters, I'm going to use these


temporary parameters here to do another round of forward prediction, which gives me my final


estimate, I could probably repeat this again. And or I could do multiple steps right here,


I could probably do a lot of things. But this is sort of the simplest case. And then I will return


these, what do I do with them? You can see right here, this is my output. Now I'm going to input


these things into what's called the task loss. And the task loss in our case here is just the video


prediction loss. So that's going to be some l two distance between the frames, the output and the


frames that actually so that these are the output frames, these are the frames that are actually


in the video. And then I'm going to just run backprop on that. So update the parameters of


both g and f on the task loss. So what does it mean? g is going to be updated, such that if I do


this whole sequence again, if I do the whole sequence of predicting, then tailoring my loss


to g, right, I tailor my loss to the g function, g is going to be updated such that next time,


if I tailor my loss to it, it's going to lead to a better outcome overall. And f is going to be


updated similarly, it's going to be updated such that, well, next time, if I do this whole procedure


of first predicting these, which I'm going to use the parameters, then updating the parameters,


and then so then updating the parameters using g and then predicting again, I update my f such


that this whole procedure will result in a better loss. This is a I think this is the magic of our


backpropagation of frameworks that we can even think of these types of things because I mean,


behold, actually writing this down and implementing the back backwards pass here yourself,


that'd be crazy. So this is the entire the entire algorithm right here. Now, again, given that there


are, as you can see, some hyper parameters here, such as the learning rates, they only do one


gradient step as we as we mentioned. So this isn't an exact enforcement of that constraint, right?


This is only an approximate enforcement, essentially, essentially, the only condition,


the only additional constraint that we introduce here is this requirement that the g function


is the same g function on all the forward predicted things. And that is our knowledge that


we are dealing with a dynamical system. And in this dynamical system, some quantities should


be preserved. The way we build the losses means that g can simply output a constant value,


otherwise, it would not be useful to the loss, right. But also the way we build the loss means


that it is not an exact constraint, like we would build this into the architecture that a quantity


must be conserved. So it's able to deal with, you know, real world data, such as this video,


where even sometimes like a hand may come in, there's friction and so on, it's not an exactly


conserving system, right. And the way we do this in the moment in the forward pass update using


this neutral loss, that means that I can now tailor whatever like I can, I can tailor the


inductive bias for this particular sample. So I can learn it's kind of meta learning thing, right?


What I learned is how to in the moment, adjust my loss function to this particular sample of data.


Now, as I said, obviously, if you had more data and all, maybe you wouldn't need this, but it does


help a lot in their experiments in the in these regimes where you do not have a lot of data,


they have a theoretical section right here, where they have a reduced case and show that it can be


useful to impose these constraints. Then they have a bunch of experimental settings, among other


things, they also they don't only do what I just said with the video prediction, but they also do


a prediction where they don't, not everything is a neural network. So where the things they predict


are actual physical quantities, and they do it using symbolic regression. And this is the same


method except it's not neural networks, it's symbolic regression. And what that does is,


it comes up with these equations, for example, for the ideal pendulum, as you can see, these


equations are insanely close, like they recover the correct equations. And these are symbolic


regressions. So the it's not you don't you didn't only have to come up with the number right here,


you actually the network had to come up not the network, the system had to come up with the entire


equation, given some basic building blocks of variables, and you can square stuff and you can


take the cosine of stuff. So these experiments show that the method can indeed recover physical


quantities that are conserved if you present them with a scenario where this is the case,


and they use either ideal scenarios, so ideal data generation, but they also use real world data from


pendulums, where obviously you have energy dissipating. And then you can you can compare. So


here, I believe they do compare with what they say is a baseline. So as that predicts into the future,


the longer prediction they do, the worse that gets. Or I guess the losses over here, you can see that.


But then also, the Hamiltonian neural networks, which enforce exact constraints, they enforce


the quantities to be preserved exactly. If you face them with real world data, you can see right


here, the quantities aren't changed at all, yet the loss still goes up because the quantity isn't


actually conserved in the real data. And the neural networks do follow the ground truth data


much more closely, because they can model also in exact constraints and not not super strict


enforcement of these constraints, which is what I think we need in real world data. They do have


a bunch of other experiments, especially as I said, also a video prediction where they do


outperform various baselines, they investigate where the network pays attention to and whether


or not you can actually move or do a lot more inner iteration steps than just one because we


just did one inner iteration steps there, there is no reason why this should remain at one. And


here they show that even though they only trained with one at inference time, they can actually take


a bunch more, and the outer loss will still go down. So this all validates a little bit of the


reasoning behind the method. Yeah, I don't want to take up too much of your time right here because


I want to jump into the interview. I let me know what you think of these more interviewee style


paper reviews. I quite enjoyed the interview. And I do think it's pretty useful to have the authors


there because they they can correct me pretty instantly. All right, see you over there. Okay,


cool. Hi, everyone. Today I have with me Ferran Alet, who is one of the primary authors of the


Nutter Networks paper, and here to discuss with us probably a little bit about the intrinsics of


the paper. And maybe also for me personally, because the paper is very technical. It's a new


field for me as well, connecting physics to machine learning, building all of this into neural


networks. There's also a bit of symbolic regression in there. So I feel a lot of things are coming


together here. I found the paper pretty cool. And it's new. And that's what's interesting. So


Ferran, thank you very much for being here. Yeah, thanks for the invitation. Wonderful to be here.


Thanks. So your your paper deals with do you call it Nutter Networks, how do you how do you


pronounce? I pronounce it Nutter Networks, but I'm not German. So I'm not sure I'm pronouncing it


properly. I'm not a German either. But but I think that the author was called was called Nutter.


Yeah. So you're pronouncing it more properly than I am. Maybe. But essentially, could you give us


maybe just first an insight? Where does the name because the name is kind of distinct, right?


Because there is the Nutter theorem. Yeah. What does the Nutter theorem say in general? Yeah. So


the Nutter theorem was kind of the inspiration for for for our work. And the intuition is that


for every symmetry of a dynamical system, there is a certain conservation law that's going to


apply to that system. So for instance, imagine you're you have a planetary system of planets


moving around. The physics laws don't change from today to tomorrow. That means that there's a time


symmetry of the system. And here, another theorem tells you Oh, if that if there is a symmetry here,


that means that there must be a quantity that's conserved over time. And in this case,


for time symmetry, there is energy that's being conserved. So we use that as a motivation, not


that the technical details more like the higher level message of the of the theorem to build a


new machine learning model. And the intuition is that in machine learning, symmetries are one of


the core ways in which we've improved data efficiency and model performance. And so it


would be very cool if we could kind of automatically learn some of these symmetries.


But symmetries are kind of hard to quantify and get a hold of computationally. And the


intuition is that they talk about kind of counterfactuals and kind of global in the sense


that when I was telling you about this time symmetry was saying, if I were to look at the


planetary system tomorrow, the laws of physics would be the same, but I don't have access to


the data for tomorrow. It's a kind of counterfactual. So if I were to look at the


it's a kind of counterfactual. So the model cannot handle this. Instead,


conserved quantities can be directly measured, I can check, oh, this quantity, which I will call


energy is being conserved on my actual data. And that makes it very easy to, to quantify.


Yeah, we've heard in, I think, in the recent past, even a lot of people attempting to get


more out of symmetries out of neural network with I'm thinking of, I'm thinking of like,


group convolutional neural networks and so on that try to actively build in symmetries into neural


networks. But it seems like they can only do that in situations where they know the symmetry that


will appear, they already know a molecule doesn't matter which way I look at it, right. So I can


directly build that in. But your reasoning is that because assessing conserved quantities


is an easier task than assessing symmetries, it might be possible to learn the conserved quantities


dynamically actually learn them from data. Is that approximately correct?


Yes, exactly. Exactly. So, and the theorem is the motivation because it tells us that


conserved quantities are kind of on the same level of powerful as symmetries for dynamical systems


in particular. If we're doing image classification that does not apply because image classification


is not a dynamical system. But that's the intuition. Yes. And you even have some some


slack in there you discuss, you know, we can we it doesn't even have to be absolutely conserved


quantity. It doesn't have to be an absolute symmetry that we deal with. By learning it from


data, we can even handle approximate symmetries. Is that right? That's a another thing that may be


a bit different from our work than other works, which is that some symmetries are only approximately


conserved or conserved quantities are only approximately conserved. So for instance,


if you have a dissipative system like in the real world, there is friction and so you actually lose


energy. If you don't consider the entire system, you'll usually have small losses. So in this case,


you would say you would like to say, oh, energy is conserved, but not quite. So it's fine if your


prediction doesn't fully conserve energy. But knowing about energy conservation maybe helps you


with the overall prediction. And maybe I want to want to get to sort of a little bit of an example


of where so people can imagine this a little bit more. Now, I only have a mouse here because I


forgot the iPad because I'm stupid. But maybe we can give the small example of a pendulum, right?


So here's a pendulum, it hangs here, and it sort of gets down here. And here is the little ball. And


the pendulum is accurately described by I think, the angle right here that it's sort of off the


off the main axis. And also its momentum, let's say it swings in this direction with a certain


with a certain speed. And this describes the pendulum. Now your model focuses on predicting


the future, let's say, or at least from from what I can tell. So what your model would be able to do


is it would be able to predict the next time step right here, right, then it's a bit here, here.


Sorry, it's a it's a little bit more up to the left, right? So it's a little bit more up,


and then it's it's even more up over here, and then it swings back and so on, it swings back over.


Now, can you explain to us what are sort of the what is the symmetry here? And what are the


conserved quantities? Yeah, so in this case, for the pendulum, we know that if we were to swing the


pendulum now, and 10 minutes from now, the physics wouldn't change. And so we know that there's a


time symmetry. And so in this case, we would say, oh, there's a time symmetry. And then another


theorem would would tell us, oh, energy is conserved. So in this case, energy is a mixture


of the kinetic energy, which is how much movement there is, and more and more movement, the more


energy and potential energy, which in this case is because of gravity. So a combination of these


must be concerned. We don't know exactly how, which formula and that's what we're going to


automatically discover. I see. And the original approach, I think would just be that here, this


arrow, I parameterize this with some neural network, right? I just say, you know, here,


I plug in neural network, I predict the next time step, and the next time step and the next time


step, and that it will maybe work, right? But it will, let's say will only implicitly make use,


it will not actually make use of the fact that something is conserved. So you go ahead and you


say, since this is a dynamical system, we know more about the system, we can impose additional


constraints. And the additional constraints right here, if I see this correctly, essentially, at


every time step, you say, I want to build a neural network that's always going to be the same neural


network that takes a state, let's say the pendulum in this state, and predicts a quantity. Let's


call that no G is the name of the network. Let's call the quantity, I don't know, alpha. And I want


to use that same neural network in all the different states that I find this thing in. And it


always needs to predict the same thing, right? Since, since it needs to figure out a quantity


that is conserved. And now, it is, it is, if I just train a neural network to always predict the


same number right here, I would just end up with a neural network that is predicting some some kind


of a constant, right? So your method figures out how do I need to build, first of all, this predictive


neural network to predict this conserved quantity, such that it actually predicts something useful,


but then also, how do I make this network right here actually use the fact that this other network


predicts common quantities, right? Yeah, exactly, exactly. So that's why the name that the word


useful in our title, because there is many conserved quantities that are kind of not useful.


And so we want to find those that are helpful for loss, final loss. So in machine learning,


we usually care about some performance, whatever it is. And so that's exactly what we that our


objective just cares about that. And the useful quantities are just a proxy and intermediate


thing for getting us to better performance. Yeah. And so here you have you have this main


diagram, I think that that would be considered the main diagram describing your method. And this is


on a task that is a video prediction task. And it's about sliding something down an incline,


could you maybe maybe describe what the task here is? It's the frames are a bit a bit low resolution.


So yes. So this is the physics one on one data set from Josh Tenenbaum's group, I think Jaijun


was the first author, and they have a collection of videos. And in this case, it's a they have a


hand dropping an object passively like it just lets it drop down and the object falls down. And


there's a second object at the end of the ramp, they collide. And then the other one sometimes


depending on the masses and the friction and whatnot, the dynamics are kind of can change.


And does so that there are multiple videos? Yes. And it's always different objects or


like some objects could be common between videos, but there's lots of objects. So it's not always


the same object. And that's the point the fact that it can vary. So one nice thing about the


other networks is that they can deal with with raw video. So some usually conserve quantities,


you get them from kind of state data. Like when I was telling when we were talking about the


pendulum, it's kind of you have the exact position of the pendulum, you have the momentum of the


pendulum, you don't have a pixel video of the pendulum. And here because we deal with neural


networks that predict the conserved quantities, you can you can hopefully get conserved quantities


from video. Yeah. So here the the diagram shows a little bit of of what you're what you are trying


to do, but also what you're trying to avoid. So the bottom path right here, if I see this correctly,


that would be if I did nothing else, except the bottom path, I would build this neural network to


just predict sort of the the future time steps. And that often turns out poorly. I don't know,


this is a quite a pixel ish mess. But it's sort of it's sort of all of a sudden, there are like


three objects instead of two. And the one is the one is kind of gone or split up. And it's a it's


a bit of a mess. And you attribute this to the fact that it's just a video prediction? Or?


Yeah, well, in this case, to analyze it and to make the problem challenging, we made the like


there was very few data. In general, you can it's all like symmetries and inductive biases are going


to be most useful when the problem is hard, and then there is like less data. So in this case,


there was a few ones of videos. And also because video prediction is pretty long. So at the very


few like the beginning of the frames, like the first few frames, there was not that much mistakes.


But when you go very far into the future, then it's a much harder. So those two problems,


lack of data and the fact that you go a lot into the future, your method is and you also have an


algorithm described somewhere, it's a bit of a it's a it's a algorithm that is right here. It's


an algorithm that has multiple steps in it. And one special part is that you have this sort of


inner optimization loop right here. Now, I want to maybe go back to the diagram. And let's go let's


walk through it once before we before we you know, take a look at the formulas and all we can walk


through it once. So the first thing that happens if I understand correctly is you take your first


input, and you do exactly what we just said, you run it through a forward prediction neural network


that just tries to predict the future. Just plain by itself. Yeah, right. So this has this has a bit


of a of a default thing. But now you try to improve that. And this is all this is the entire thing


we're describing right now that is one forward pass through your system. So if you would take


every single prediction that you made, and you would feed it through this G network right here,


and this G network is you call it an embedding network. That is the thing ultimately that's


trying to predict a conserved quantity. Yeah. But it's not it's not necessarily just outputting one


number, it's outputting an entire vector. Yes. So it's an outputting an embedding vector. And the


the goal obviously is that for all of these inputs, it should output the same embedding vector.


Mm hmm. Exactly. Exactly. But so ah, so but this is this is going to be, let's say, trained such


that across the data set, it works well. So maybe, you know, for this video sequence, it's going to


predict approximately the vector a for all the frames, if it works well. And for another sequence


with two different objects that obviously have a different total energy or so, it might predict a


different embedding vector. Exactly. But all the same across the across the video sequence. Okay,


so this is how we can imagine you train this G network to sort of predict whatever is special


about this particular data point, but inside of the data point conserved among all the frames.


Exactly. Because if it was the same a for everyone, then you would have the issue that you


mentioned at the beginning, then it's a useless conserved quantity. Yeah, yeah. So it's, it's


almost like a bit of a description of the scene as such, right? That makes the video predictors


life easier, if you have sort of this this global description. Yeah, yeah. So the intuition, I


think is let's think about when the if the network G was very good at predicting the conserved


quantities and perfectly told you all these five quantities, I know for certain that they're going


to be conserved, then we could, we will see the next step. We haven't gone through it yet. But the


intuition is that knowing these five conserved quantities is going to tell me a bit about what


my prediction should be. Yeah. And so it's kind of free information that I get to know about


constraints. So it's kind of a non supervised loss that I have access at this time. Yeah. It


restricts it restricts what you can output right? Because ideally, the F network should only output


whatever the G network says is is the same, right? If the F network can only output things that the


G network will embed to the same place in the embedding space or a similar place. Yes, there's


just to be 100% precise, there is lots of images that could make the network G happy, because it


only constraints like a few dimensions. But it has to make the network G say, Oh, this is


approximately what you had at the beginning. Yeah. Okay. And so that that comes in in the next step.


So here, what you do, you use you take the input again, and you route it through this F network


again, but now this F network doesn't is not like a freeform predictor, but it actually takes has


somehow the notion of, of this information that the G network output out of the initial sequence


again, and you do this in a very special way in that you actually take the parameters of F and


you update them on the fly. Yes, you update them on the so this is within a forward pass, you


actually update the parameters into the direction of the gradient of G. Exactly. Yes. So yeah,


sorry, this is I think that that it takes it. Yeah. So here, you have this neutral loss. Yes,


exactly. Which do you maybe want to talk about this briefly? Yes. So about another loss? Yeah,


sure. So the other loss essentially is telling you you should have you should conserve G.


So the you know, for a fact that so there's two ways of conserving G, they're roughly equivalent


if you fully impose them, if you don't fully impose them, they're not equivalent. That's why


we put the approximate sign. So let's look at the term A here. It's basically saying, oh,


you should conserve G. And so it should be all of them should be equal to what G was telling you


for the input x naught. So if you make the embedding of your prediction, note that the x


of t has kind of a tilde on top of it. So your prediction for x t should have the same conserved


quantities as your input. And that's what the first term is and just an MSC over this neural


embedding. The second one is very similar. Sometimes it's a bit more useful, more stable,


because instead of if instead of comparing to your the very beginning, you compare it to the


previous time step, you have a more immediate signal and you basically say you should conserve


it. Every time you apply F, you should conserve G. So that's basically important observation.


And now we update theta and theta are the parameters of F, right? These are the parameters


of we update these on the fly. And I suppose that we just do this, you know, in the moment,


and for the next data point, we go back to the sort of original parameters. Yes. And do this


again. So this is sort of an on the fly update for, you know, a temporary update of these


parameters into the direction of this quantity right here. So this is the gradient of exactly


the loss that we just discussed with respect to the parameters of F. So essentially, it says,


what parameters would make F more apt at fulfilling this loss, which essentially means that these,


which how do we need to change F such that these forward predictions make the G conservation


happier? Exactly, exactly. So this is some previous work of ours, which we call tailoring. And the


idea of tailoring is just because of what you said that the fact that the addition is customized


for each individual data point. And the idea there was a general way of encoding inductive biases


with unsupervised auxiliary losses. So auxiliary losses in general, you say, for instance, one thing we could say is, oh, why not we add energy conservation when we train? Sometimes auxiliary losses would say, okay, I train for good predictions. And I train for energy conservation at training time. But if you do that, you're not going to enforce energy conservation at test time. Because at the test time, you're going to have a generalization gap in energy conservation. But energy conservation is not going to be a problem.


Energy conservation or any type of conservation or any auxiliary loss can be checked before making the


prediction at test time or at training time. Inside the prediction function, I can first make my prediction and see, okay, do I like it? Does my auxiliary loss, does my unsupervised loss like this prediction? And if not, I can take a gradient step or multiple gradient


steps to improve my unsupervised loss, in this case, the conservation loss. And so this makes it much better for the particular point we care about, which is the one we are making a prediction for. It's a bit surprising because it's a single data point. And maybe you have trained with a million data points. So the question is, why? Why does one data point matter if we've trained with 1 million data points? Well, the idea is that you're training on the exact point you care about. So enforcing inductive bias in the exact point you care about right now, for


which you're making the prediction is going to have a very big impact. And so in this case, this gradient step improves the prediction just for that one point.


Yeah, maybe it's it's also important to highlight that the parameter here, this theta that we start with, and also the parameters of G, those are the ones that will be learned during the training procedure across the entire training data set. And then the parameters


here, those are always constructed in the moment data point by data point to, as you say, tailor the inductive bias and the inductive bias, in this case would sort of be that this entire term right here, exactly, essentially says, you know, what do how do I need to change my predictor in order to conserve the particular thing that G decides is the common quantity for this data point?


Yeah, yeah.


And, and this gives rise to the the algorithm. So here is what we just discussed. This is the forward forward prediction sequence with this inner optimization step. So we first predict this plane sequence, then we temporarily update the parameters. And that allows us to again do the forward pass, but now with the updated f function. And that gives us sort of our final predictions.


And as you can see here, during the training, we, we sample always batches, we forward predict using this inner update. And then we take outer gradients. And the L task here, that would just be that what you call the task loss, this would be the video prediction loss, or something like this. Okay. So my I have, I have a lot of questions.


First of all, this, it seems it seems quite intricate, right? Because if I think, okay, these outer gradients right here, especially this gradient right here, this is, how do I need to change theta? Now, okay, how do I need to change theta? This depends on these predictions right here. These predictions right here, have one forward pass using theta, then have a gradient with respect to theta right here. So that's the first thing that we need to do.


Right. And, and the all of those come from this quantity, which is already a forward pass using theta, right? Is is this actually how it's implemented in practice? Do you do stop gradient somewhere? Do you have any hacks? Or is this actually because it seems mighty unstable, right? How does this actually work as you specify?


Okay, yeah, that's a good question. So in general, it depends. So if it was a single prediction, so if it was like the default, sometimes we've applied this kind of prediction time optimization, the data in procedure to regular tasks like image classification, I think like this, it's not that unstable, because if you're just kind of doubling that computation graph, because you make one prediction, and then gradient step and then double that prediction. So that's fine.


Now, here, you have two issues, the fact that you're taking the green step and, and the fact that you have many predictions, many predictions that kind of build upon one upon the other. So that could get could get tricky. In practice, we've seen that if the overall training regime is stable, then it works fine. But if the overall thing is already unstable, then it's unstable. So that's a good question.


But if the overall thing is already unstable, then it's extremely tricky to to add things there. So for instance, one thing we realized was that because video prediction is very expensive. And basically, we couldn't fit that many examples on a GPU, literally, I think two or four. So we were initially using by normalization. And so that was making the training the vanilla training of the vanilla neural network. So just f already unstable.


And when we were adding our another network improvement on top of it, it couldn't learn anything. We'd swap the batch normalization for layer normalization, then the vanilla training was very, very stable. And then suddenly, the neural networks worked out of the box. And we think that that's because if the grid of the original gradients because of the batch normalization, if you compute the batch statistic with a very small batch, it's already very crazy unstable. And then we couldn't learn when


the other thing is already stable, then it seems for us it worked pretty out of the box when we swapped the layer normalization.


Okay, that sounds good. Yeah, I would I would expect so.


Yeah, so for instance, I would expect for instance, if we were to do 100 steps or many more steps or, for instance, we were discussing before how there were two losses, that sometimes we tried one or the other.


The reason we came up with a second loss that conserves kind of the conserved quantity between this time step and the next time step was when we were using batch normalization, we were wondering, oh, is our another network unstable?


Or and then we realized, okay, no, it's that it's the vanilla network that was unstable. But that was part of our concern. Because there is some papers that mentioned that when there's a when you're back propagating through a very deep


graph, then the gradients are sometimes not very informative. In our case, we found that when the thing is pretty stable, seems to work fine. But I could expect that if you make very, very long predictions, or your thing is already unstable, then it only adds to the instability taking the second order.


Yeah.


Yeah, another thing that struck me is that there is only right, there's only one gradient step here. You take one gradient step, and I'm going to, yeah, that that might also be something where stability or computational graph size. First of all, you just do a gradient step, many things would be possible, right? You could do an adagrad step, you could do an Adam step, you could do a gradient step.


You could do an adagrad step, you could do an Adam step, you could do a line search or a Newton step or anything like this. But you have chosen to do like the most simple thing, which is a single gradient step, right?


I think the key word here is what you said about simple. We could have done anything else. But I think simplicity is something to value a lot in research, I feel. And so we went for the simplest thing. And so one gradient step. And if you can train with three gradient steps, and we've sometimes done that, it's a bit better because this allows you to take smaller gradient steps.


And then sometimes you optimize the inner loss further better. But in terms of one, simplicity, if it works with one, it's better. And two, especially when you present the algorithm in a paper, you really want to show the simplest version.


And then usually people now know that, okay, if you can take one grand step, you can usually take more than one grand step. And it will just make the competition graph larger, but that's fine. So we were striving for simplicity, both when we were implementing and then when we were showing the algorithm.


And you do have experiments that show that even though you learn with one gradient step, and that is down here somewhere, even though you learn with one gradient step, you can in fact, at inference time, then perform more than one gradient step. And that up to a sizable amount of steps, like up to 100 steps or so here will actually improve the outer loss.


Yes, yes. We think that essentially the inner loss is kind of a projection loss, right? Because you keep saying, okay, why don't you make G happier and happier? And especially in the theory section, we go a bit about this, but essentially, there is many futures you could have predicted and some of them make G higher.


Imagine it's only one quantity for now. Some of them will make G higher, some of them will make G lower. And when you force to conserve G, all these futures say, okay, no, you should conserve G and therefore it's kind of projecting one dimension. And so in particular for conserve quantities, applying the same laws over and over, it's kind of stable because you will just keep going closer to these manifold of predictions that conserve G.


Yeah. So there's no, let's say, danger of overdoing. I mean, there's a little bit, but as I said, it hits after like 100 steps, which is quite a bit, right? Given that you train with one.


Yes. So eventually, especially because also these are neural networks, so it's not like it's a, for instance, if we've tried with this with hard-coded losses in the previous tailoring paper, and it's the true conserved quantity and the energy is truly conserved, then you can freely do that and it will keep going down.


But because it's a neural network, then suddenly, I think you're going outside, it's kind of a distribution shift. You train G to be useful for one or two or three green steps, now you're using it for 100, it doesn't make you any promises.


Yeah, that makes sense. Now, so I wanted to also come back a little bit to a more conceptual idea. Maybe this is also a bit of a question about tailoring in general, what you do here that you essentially adjust the parameters of your forward predictor on the fly.


There are many ways you could have combined the two networks, right? The one network that essentially predicts the conserved quantity, and the other one that forward predicts, for example, you could have optimized the predictions themselves at runtime.


Yes.


You could have made both of them happy. You could have, I don't know, you could have just learned it as one thing and not even bothered with runtime optimization. Why did you choose this tailoring approach in particular? It seems a bit cumbersome, right? And it's not maybe the first choice one would come up with.


What are the advantages here?


There's two things in your question. Let me answer one after the other. So there is one, why the prediction time procedure, the runtime procedure, and then the other one is why adapt theta instead of x.


So let me start why the runtime procedure. It goes back to what we were talking a bit like 10 minutes ago or so.


The fact that the alternative to tailoring is auxiliary losses, which are, you could say, okay, we are going to learn an auxiliary loss that is going to be helpful for the final prediction.


So there's two points here that I think would be improved.


The first one is we are trying to learn an inductive bias.


So, for instance, one very cool thing about Hamiltonian neural networks or CNNs or transformers is that the inductive bias that they encode into the network applies at training time, but also applies at test time.


So you know that you have equivariance at test time and you know that your prediction satisfied is inductive bias.


And so, auxiliary losses, if you train for energy conservation or whatever laws you want, do not enforce, do not satisfy inductive bias.


And so for it to be a proper inductive bias, it has to be satisfied also at test time, and that's why we optimize it at runtime.


You also have to optimize it at training time because if you optimize it only at test time, then you have a distribution shift.


So that's why it has to be optimized inside the prediction function.


So that's the first reason why. To be a proper inductive bias, it has to be optimized at runtime.


The second question was, oh, sorry, and there's a second reason why we also do that instead of auxiliary losses.


The reason is that there is a very immediate signal. So imagine you encode energy conservation at training time,


then it's a very loose signal to the final test prediction because you're saying, OK, this is going to affect my final training parameters,


and then I'm going to use my training parameters on a validation set, and this is going to lead me to good predictions.


But this only happens, you only can look at the effect at the very end of training, and then you're going to use that on validation.


And so you could do that, and I think there's people that do that using implicit gradients, but the signal is much, much more cumbersome.