the image net has now become, you know, like speedrunning, there is there's glitchless,


NFNets: High-Performance Large-Scale Image Recognition Without Normalization (ML Paper Explained)

and the equivalent is like additional training data less, and so on. In any case, we'll go through


the paper, we'll discuss what the tricks are to get the normalizer free networks to work,


I do have also a fair bit of, let's say, criticism against this paper right here. But in general,


it's a pretty cool paper, the code is available, of course, link to the code, you can try it out


yourselves. And that's, you know, it's pretty cool that the code is available. All right.


If you like content like this, as always, don't hesitate to share it out, consider subscribing,


let's dive in. What's the problem with batch norm, batch norm? As you might know, I've done a video


on batch norm, but essentially, what it says is that if you have a data point that goes through


a network, you know, it will experience various transformations as it goes down the layers.


However, some of these transformations are quite unfortunate if you built the network a little bit


in a wrong way. So what might happen is that your initial data distribution might be, you know,


in machine learning, it's good practice to center the data and around the mean and kind of, you


know, scale it to unit variants or something like this. But then as you progress through the layers,


and especially if you have something like relu layers, they only extract the positive part of


the signal. So with time, it can happen that the intermediate representation right here, for


example, is, you know, something like this. So it's very skewed, it's not centered, and so on. And


the current methods we have in machine learning, they just work better if your data is sort of


well behaved as a nice condition number is centered and so on. So what batch norm does is


every layer it comes in, it looks at the current batch of data, the current mini batch, and it


centers and rescales it. So what it would do is it would transform this data by a simple


standardization procedure into a well behaved data set. Of course, remembering the transformation for


a back prop, and then feeding that data to the next layer. That's batch norm. And it has several


disadvantages. So the disadvantages of batch norm, this paper identifies three batch normalization


has three significant practical disadvantages. First, it is a surprisingly expensive computational


primitive, which incurs memory overhead, okay, which is, you know, you need to compute these


means, and these scalings, and you need to remember them for the back prop. All right,


second of all, sorry, significantly increases the time required to evaluate the gradient in some


networks. I mean, there is Yeah, there is some back prop you have to do through all of this


standardization. Second, it introduces a discrepancy between the behavior of the model


during training and at inference time, which is also true because at inference time, you don't


want this kind of batch dependence, you want to be able to feed a batch of data to the next layer,


you want to be able to feed a single data point, and the result should always be the same


irrespective of the other data. And people usually do this by so at training time, you simply


calculate this mean shift right here and the scaling that you have to do. And what you would


do is you'd have kind of a database, a special buffer where you save these things for every


batch. And then at test time, you simply look at your buffer, you kind of build a mean a moving


average over your training data, and you'll simply use those shifts and variants. So you have a


discrepancy between training data, which just looks at the current batch and inference, which


looks at your mean your average over the last few batches. And third of all, and this is the so this


introduces hidden hyper parameters that have to be tuned, which is kind of how fast the mean decays


in your database. And third, most importantly, so most importantly, batch normalization breaks the


independence between training examples in the mini batch. So not you, it now matters which other


examples are in the batch. And that has two consequences. So the first consequence is that


batch size matters. So batch size matters in batch normalization. If you have a large batch,


you can compute these means of the data, they are a much better approximation to the true mean


of the current data set at this particular representation, then a small batch. So if you


just have three examples, the mean is going to be a very noisy approximation. Whereas if you have


a large batch, it's a good approximation. So batch size matters for batch norm. And second of all,


so distributed training, distributed training, yeah, distributed training becomes extremely


cumbersome. Because if you do, for example, data parallelism, which means that here you have your


batch of data. And we know for some applications that large batches are pretty favorable for


training, they stabilize training, you can do larger step sizes, and so on. So what people do


is they split the batch, they shard one batch into, let's say, three different parts. And they


have the network on three different machines. So the same network is on three different machines.


And what you would like to do is you would like to forward propagate all of these batches through


the network, sorry, this whole batch in three different shards through the network, and then


back propagate and sort of communicate the gradients around. But now imagine if you have a


batch norm layer. So if you have a batch norm layer right here, it's going to be the same here.


And it's going to be the same here. What you would have to do technically is you have to forward


propagate the signal right here to the batch norm layer. And then you'd have to communicate


these batch statistics between the batch norm layers, because otherwise, you don't have the


mean and the variance over your whole batch that you feed in, right, you can opt to not do this


computation. But then again, you run into the problem that usually these, the number of samples


in the shard is fairly small, and you have a bad approximation. So batch norm just kind of makes


certain things complicated, right. And this interdependence of training data points is one


of those things, and they call it the most important things. So they say this third property


has negative range of negative consequences. Practitioners have found that batch normalized


networks often difficult to replicate precisely on different hardware. Batch norm is a very


common term in training. So let's say you have a batch normalization, the cause of subtle


implementation errors. Okay, well, yeah, especially during distributed training.


And then it cannot be used for some tasks since the interaction between training examples in a


batch enables the network to cheat certain loss functions. So this is, let's say you have a like


a time series prediction, right. And in a time series prediction, so you have your your time


series. So what you usually do is you say, well, this is my input. And this is my goal. And then,


and this is my input. And this is my goal. So it's kind of it's like language modeling if you do


that. So you want to slice one sequence into many training samples. So you do like overlapping


training samples are like, this is the input. And this is the goal. Now, imagine you have those two


in the same batch, then technically, the this training sample here could just kind of by means


of the batch statistic aggregation, information can actually flow because this here technically


is part of the input of one training data point, but it's the label for the other training data


point. So there can be information leakage in that. So you shouldn't use batch norm or anything


that connects the training samples to each other in these particular cases, it's kind of an edge


case. And you can, you can probably get around it by just having a big data set and shuffling


a lot, but still, so they say they solve all of these things. Specifically,


they say, we propose adaptive gradient clipping, which clips gradients based on their unit wise


ratio of gradient norms to parameter norms. And we demonstrate that AGC allows us to train


normalizer free networks with larger batch sizes and stronger data augmentations. So their method


of of circumventing batch norm of building networks that don't have batch norm anymore,


is going to be this adaptive gradient clipping, it's going to be in combination with earlier work


from an earlier paper that they've done. And but this paper introduces specifically that


active gradient clipping, you're going to see it's a pretty simple idea, it should be implementable


in pretty much any network out there. And it has the potential to become kind of a staple


component in deep learning, if it turns out to actually work as well, as they say in the paper.


They say we design a family of normalizer free resnets called NF nets, which set the new state


of the art validation accuracies on ImageNet for a range of training latencies. Okay, so they repeat


these things from what I said in the intro. And they also say achieve substantially higher


validation accuracy than batch normalized networks when fine tuning on ImageNet after pre training,


so they also have a good transfer accuracy. Now, my first problem with this is that


the two things here are kind of not very related. So the gradient clipping is an actual, let's say,


a contribution, it's a new method, they suggest that they measure it absolutely cool. But then


they go around, and they do like giant architecture searches for how could we replace the ConvNet


block and so on, to come up with these NF nets, which is also cool. But it is not clear to me


that these two things are necessarily as connected as they make it to be, of course, they would say,


well, since it's normalizer free, we can build up but I don't see why you couldn't just do like


better architecture search for classic batch norms networks. So it seems like


and then you don't you don't know where the gains actually come from, like whether or not you need


the gradient clipping or whether the contribution here is actually to figure out a kind of a better


ResNet architecture. You know, who, who knows? In any case, they the structure of the paper is the


follows, they first go, what does batch norm do? What does it do well? And then how can we replace


all of the things that it does well, by our own stuff and then not need batch norm anymore. So they


identify four things, batch normalization downscales the residual branch. So in a ResNet,


you usually have an input. And then you put that through a series of layers to the output. But


first, you add the input again, so you add the two. And this and this is, so this part is called


the residual branch, it's kind of, so this is the identity function, I've done a video on ResNets,


if you want to learn more about that on residual networks. And batch norm will downscale the


residual branch implicitly. And that just means that the signal strength is more in favor of this


identity function, which is the entire point of ResNets, which is the whole point of the


entire point of ResNets, which makes training more stable. Second, batch normalization eliminates


mean shift. And that's the thing we said before, that for example, if you have relu's or something


like this, they only retain the positive part of the signal, which leads down the network to quite


a shift in the mean of the data. And batch norm eliminates that. Third, batch normalization has a


regularizing effect by means of the, the batch statistics are noisy, which, you know, we said is


a problem for inference. Yes, but it is also has a regularizing effect during training. And lastly,


batch normalization allows efficient large batch training. So it smoothens the loss landscape. And


this increases the largest stable learning rate. Okay, so we want to get we want to get to a point


where we get all these benefits, but don't need batch norm anymore. So first, they introduce


their old paper and their old paper, it's not that old, I think it's so it is this one here,


you can see it's also this year, it's an it's it's an iClear paper. And there, they built these


normalizer free ResNets, these NF ResNets, not to be confused with NF Nets, which this paper


introduces, okay. So the normalizer free ResNets already tried to build normalizer free ResNets,


they manage, they manage to build, you know, networks that train, but they don't beat the


efficient net efficiency yet. What they do, specifically, is they just pay attention a lot


to scaling. So they introduce, for example, these parameters, alpha and beta. And what they do is,


essentially, in every single block in the neural network, they try to very carefully predict how


this block will change the variance of the data. And then they build constants here. So this is,


is this alpha or is this beta, I think this is alpha goes after and beta goes before,


they build constants, alpha and beta, these are constants that are made particularly for


the architecture. So if this is like a conv layer, they pay attention, and they make these constants


such that the variance kind of stays constant as you go down the network. So it's very much like


people build deep learning frameworks, where you know, for every operation, you have to define a


gradient, and then you can chain them together. Here, for every block, they, you know, carefully


think about how it affects the variance of a signal. And then they design appropriate


scalings to bring that variance back. And if you do that consistently, and it's it is quite hard,


right? And they have to do a lot of things, for example, also kind of a, a variant of weight


standardization and so on. But if you do this, then you can train quite large batch sizes. So


normalizer free resnets match the test set accuracies achieved by batch normalized pre activation


resnets on ImageNet, a batch size 124. They also significantly outperform their batch


normalized counterparts when the batch size is very small, but they perform worse than batch


normalized networks for large batch sizes. Crucially, they do not match the performance of


state of the art networks like efficient nets. And this paper is going to fix this. All right.


The main way, or one way, the thing the paper introduces is this adaptive gradient clipping.


Now what is gradient clipping? So usually, usually, right, you have a parameter, it sits here


in the parameter space, and then you get a gradient and you follow that gradient, like over here,


down here, over here, down here during training. Now sometimes, sometimes you have a batch of data


that just tells it to make a huge jump. And this, these huge jumps are often the cause for training


instability. Because, for example, if you use SGD with momentum, that thing will get into your


momentum term and just skew the training over here, it will screw with your atom buffers and


even plain SGD, it's not really good if you take giant jumps. So gradient clipping simply says


whenever a gradient of any parameter is larger than a size, let's say, this size here, we'll


simply clip it, that's, we'll scale it. So that's the maximum length. So if it is, if it is, you


know, if it's a good gradient, we're surely going to see it again. But if it's a bad gradient, we


want to limit its impact. The problem is that it's very sensitive to this parameter right here.


And the reason is, it's not adaptive. So what do they mean by adaptive? What they do is the


following, it's almost the same. So as you can see, G is the gradient. So this part right here


is the same, you want to scale the gradient, but you want to not only clip the gradient to its own


norm, but you want to clip the gradient to the ratio to this ratio right here. So the ratio is


going to be how large the gradient is, versus how large the weight that the gradient acts upon is.


So if you have a small weight, if you have like a small weight, and you suggest a small change to


it, fine. But if you suggest a big change to the weight, then it's like, I'd rather sorry, I


probably should draw this like this. So small change, fine, large change, not so fine. However,


if you already start with a large weight, then you know, large changes might be appropriate,


because that's the general scale of that weight. It is though, it is an approximation, right? It is


not, it is not a it is not the end all, it's simply a good heuristic, because you can make cases


where just comparing these norms don't mean everything. So if your weight is this, and


you have kind of a gradient that's really large, that goes into this direction, you know, that


might be bad, because you kind of scale the gradient by a factor of three right here. But if


I take the same length gradient, and just put it into the other direction, you've not scaled the


weight at all, basically, but it's the same length of gradient. So just looking at norms isn't


everything, but it seems to be a good heuristic. And with that heuristic, a lot of the problems


of batch norm fall away. So they do ablations right here, where you can see that, for example,


if you compare batch norm networks, the normalizer free resonance from the last paper and the


normalizer free ResNet, plus this adaptive gradient clipping, you can see that after a


certain batch size, the non AGC network simply collapses while the ones while the batch norm one


and the gradient clipping one prevail. So this seems to be the recipe to go to higher batch sizes,


pretty, pretty cool. But over here, you can see, here is a different thing here, it's top one


accuracy versus clipping threshold. So where where do you set? Of course, there is still this


parameter here. And they complain that it's very finicky with the if you don't do adaptive gradient


clipping, so I expect this to not be as crucial if you do non adaptive gradient clipping. However,


here, you can see that it has a crucial dependence on the batch size of all things. So you can see


at small batch sizes, you can get away with clipping at a pretty large threshold. But then


at large batch sizes, you can see you have to, you have to keep the threshold pretty low because if


you clip it higher, then it's you know, it collapses. Now, I was told that one of the


problems with batch norm is this dependence of training data points amount like to each other.


And I kind of expected this paper to fix it, but it doesn't in a very subtle way. So here is how


here is how the gradient clipping works. I told you right here, if the gradients too large, we're


going to clip it, right? Pretty simple. If it's too large, you know, just clip it down. But what


is a gradient? A gradient is actually composed of the batch of data that you feed through, right?


So you feed a batch of data through a network, da da da da da. And then you have a weight somewhere


here. And the gradient that you get for the weight, so maybe the weight is here in weight


space, the gradient you get for the weight is an S sum. So your gradient for your weight of f of x


is going to be so this is a large x, this is all the data is going to be a sum over your data points


of the gradient now with respect to that, because your loss, sorry, this is a loss function


that your loss is a sum. So your gradient is the gradient of a sum of loss functions.


And these are interchangeable. Don't come at me math people, not always, but in this case, I guess.


So I hope you can you can sort of see that your gradient is going to be a sum over data points or


a mean over data points. And that means that it's not actually one gradient, this one gradient is


made up by many, many data points pulling that weight in different directions. And the gradient


you end up with is simply the average over or the sum over all these gradients that the individual


weights put it. So if you now think in this in terms of gradient clipping, you can see that


gradient clipping, and you think that during the data, data feeding process during the training


process, every data point is an sort of an estimate of the whole data set. That means that


your gradient is going to be noisy. That's the point of SGD. What happens to noise if you average


it over a bunch of iid samples, it gets smaller in relation to the signal, right? If you have if you


input the whole data set, you have no noise, you have a perfect gradient, at least over your training


data, as you make the batch smaller and smaller, you have more noise. So if you clip on the final


gradient, as opposed to the individual data points, and I've checked in the code, they first do the


sum or the average, then they do the clipping. If you do that, that means now the effect of the


clipping is going to be dependent on the batch size. And it means that you implicitly interconnect


your training data, because if you have a noisy process, right, so if this is your, this is your


base noisy process, and you average, you'd always sample two things from that from the noisy process,


it has this much noise, you're going to get something that has less noise, because it's the


average of two things. Now, if you average over 1000 samples, you're going to get something that


has very little noise, right, every now and then it has a bit of noise. What you want to do with


the gradient clipping is you want to limit the impact of bad training data points, training data


points that just tell you to go a lot into a bad direction. What does that mean? If I have one bad


training data point in my batch of four, that is going to spike the gradient a lot, like right


here. So my gradient clipping can be pretty high. If I want to clip if I want to limit the impact of


that bad data point, if I have a bad data point, my gradient is going to spike pretty heavily,


and therefore my clipping threshold should be high. However, if I have one bad training data


point in 1024, it's only going to spike the the total gradient a little bit. And therefore,


in order to filter out my bad training data points, I need that threshold at a much lower level,


right. And therefore, I'm going to, you know, filter out that one here. Now. So that's what I


mean, it makes the training data points implicitly dependent on the others in the batch as batch norm


does, it just doesn't do it explicitly. But still, there is a dependence on the batch, which I guess


you could solve by doing the clipping before you do the averaging. But it's not as easily implemented


in the frameworks that we have. By the way, if you do, and if that gets you a better network,


cite the channel. Yep, on the way to become the first cited YouTube channel in a machine learning


research paper. I could be wrong, though. I mean, I've looked at the code, I could it could be that


they do it before. I don't know. Okay, so that's the deal with clipping and my issues with the fact


that this does still depend on the batch. So we haven't, we haven't saw actually solve the


dependence on the batch yet. We have probably solved the computational issue, they say, you know,


for calculating batch norm, it takes a while, and it takes lots of compute this here, it doesn't,


it still needs compute. However, probably not that much since you can still you can just do it during


the backward phase, right? You don't need anything during the forward phase for doing this clipping,


you simply during the backward phase, you need to normalize clip, and you're good. So we can take


that one. And then my third criticism right here is that they say the third or the second criticism


on batch norm is that it has different train timed behavior as test time behavior, which we


discussed, which is true. But then what does their network contain? Dropout dropout? What's the


property of dropout? It has a different behavior at train and at test time. Like, so, you know,


don't. It's, it's okay, we get that batch norm has these limitations, but your paper doesn't


necessarily make them better. It just kind of shifts them to different to different things.


Okay, enough rant. So the second part of the paper goes into architecture building. So I actually


don't want to touch this as much. But what they do is they say, well, now we go about building


a beast architecture that just outperforms everything else. And I'm not sure what it has to


do with normalizer free networks. Like, this is something you can do with or without batch norm,


but they come up with this new architecture right here, this new block. Let me scroll to the end


these new two blocks for resnets. So the right one is where you do not have a kind of a down or up


sampling. And this one is where you do. But, you know, they have done a lot of search. And you can


see here are the beta and alpha parameters to make this normalizer free. But, you know, doing


architecture search, you can do that by yourself, like you don't need the normal, maybe you need


the normalizer free, but they don't make it clear that these two things are so intimately connected.


And then they get the model they get up here. And, you know, there is quite a bit of evidence in the


paper that sorry, this one, there's quite a bit of evidence in the paper that this adaptive gradient


clipping actually has some nice properties. Yeah, it allows you to go larger, larger batch size and


so on. But again, it's it's a bit unclear what gains come from the normalizer free what gains


come from the adaptive gradient clipping and what gains simply come from the fact that they have


better architectures. So their whole point in architecture search is that efficiency net what


it tries to do is it tries to achieve an accuracy with as little as little flops as possible.


However, modern accelerators cannot necessarily make use of those, you know, savings in flops,


because you know, they have certain constraints. And therefore, this network right here, it focuses


explicitly on training latency, which means that if you use current hardware, which means GPUs or


TPUs, how fast is training. So for a given time of training, how much accuracy do you get in there,


since it's particularly built for that, as you can see, it beats efficient net by a lot. However,


if you look at this in terms of flops, they have a graph down here. So if you look at this in terms


of flops versus accuracy, as you can see, it aligns with efficient net. So the kind of line here is


pretty, as you can see, like it's pretty straight. It's as if you were to scale up the efficient net


architecture for a bit more in terms of flops. So this is better in terms of so this is more


optimized for current hardware, this kind of of networks. Yeah, so that is pretty much it they do


do a lot of ablations comparisons. And it's not like I don't believe that the adaptive gradient


clipping is, you know, does nothing or that, you know, clearly they also they always do experiments,


they compare the normalizer free res nets with the batch on res net. So they try to isolate


the individual parts. Still, I'm not sure how I feel about papers that have, you know, a lot of


different things in one paper. And then they get state of the art, you never exactly know why that


is. And the last thing I want to mention that's cool about this paper is appendix E, appendix E


show you that appendix E is negative results. And this is really cool. So here is a list of all the


stuff they tried that didn't work. And you know, it's one page, but still, it is very, very good,


even if it's only to see that other researchers try a whole lot of stuff and fail as well. So I


invite you to check out the paper, I've linked the code, you can take the code, it's in jacks, which


One Model For All The Tasks - BLIP (Author Interview)

Hello, this is an interview with the authors of the blip paper. If you haven't seen it,


I've made a review video of the paper itself. Be sure to check that out. The authors have


seen that and are directly able to respond to it. So we all start on an even footing.


It's very cool to have the authors on and this interview particularly was really interesting


to me. I hope it is to you. As always, thank you for everyone who leaves a like who leaves


a comment. Thanks to all the patreons and the support I get on on Twitter and on YouTube


itself. It's really cool. And I wish you a lot of fun. Thank you. Hey there, a quick


shout out to today's sponsor. Assembly AI is an AI company that offers accurate API's


for speech to text. As a developer, you can use these API's to automatically transcribe


and understand audio and video data in just a few lines of code. Assembly AI automatically


converts asynchronous and even live audio streams into text. They have so many features


that help you understand your audio data. For example, summarization, content moderation,


topic detection, and much more. Please check them out using the link in the description


to let them know I sent you. Now let's get on with the video.


Hi everyone. Today I'm here with Junnan Li and Dongxu Li, who are two of the researchers


of the Blip paper. It's a very big honor to have you here. Welcome both of you.


Thanks for having us.


Really happy to share our work.


Yeah, this paper was really cool. I think when it came out, everyone saw it and it generated


quite a bit of buzz because it is a new approach to incorporating images and language and it


can do a lot of things at the same time. It is a big system. And yeah, I was super happy


when I saw it. And when I read the paper, I was also pretty happy after I read the paper,


which sometimes isn't the case anymore after you read the paper. Yeah. And if you would


if you would just to dive in maybe if you would pitch your idea to someone like someone


comes to you in a poster session or so maybe for people who haven't seen the paper review


just extremely briefly. What does your paper say or what do you propose?


So maybe I can take this question. I think the major point of our paper, the setting


point is that we propose a unified framework for visual language pre training where we


can portray this model that has the capability of doing both visual language understanding


and visual language generation. So what understanding means is that it can jointly understand the


two modalities, namely image and text, and produce some kind of multimodal features that


can be used, such as for classification tasks. And what generation means here is that it


can generate text based on some image input, for example, for image captioning, is one


of a typical generation tasks. So I think this is the main idea of our our model. And


in terms of the technical in terms of how do we achieve that, I think there is one big


point that I would like to highlight is we do have this data set bootstrapping to tackle


the challenge of noisy web training data, because existing works, a lot of them pre


train on those data that are collected from the image from the web, which contains the


image and all text pairs which can be noisy. I think you mentioned in the review video.


So what we do here is that we want to see sensitively generate captions, and also to


use a filter to try to remove the noisy captions. And by doing so we can significantly improve


the quality of the data set. And I think one of the key message we want to send in the


paper is that the quality of the data really matters. It's as important as if not more


important than the quantity. So a lot of passwords have focused on scaling up the model with


big data. But here, we do scale up, but we also focus on the quality of the data.


I want to I want to dive into this data bootstrapping right away, because it is almost it is almost


a bit of an independent thing from the system itself, right? We've long known that we can


trade off quality for quantity, but usually it is in an exponential fashion. So to get


the same amount more quality, we need exponentially more data if we want to achieve it with with


less quality data. Did you was this was this which came first, the idea of building the


vision language model or the idea of filtering or the data set because they both play nicely


into one another in your paper. And I'm just a bit wondering, how did this come to be?


Which came first? Why? Why one or the other?


Yeah, so actually, for my research, for my past papers, I've focused some papers on this


weakly supervised learning or learning from the noisy data. So I've always been quite


interested in how do people train models with imperfect data, which is a very practical


scenario. And I think this is our this field may deserve more attention is not as popular


as some of the other fields. But it's really a very practical issue. And it do exist for


vision language pre training. So actually, one of my previous paper in vision language


pre training, which we call it LBATH model, it was published in New York last year, we


have this kind of self training scheme, where we want to clean the noise in a data set,


but it's in a relatively more simpler way than what we do here. So rather than generating


synthetic captions, we were doing some self distillation thing. So then we take it to


the next step in the brief paper, where we first look at the data set, and we see a lot


of noise. And here noise basically means that the caption is not really describing the visual


content of the image, it may still be a good human written text, right? It's not the text


is grammatically wrong, is grammatically correct. It's just that it's not aligned with the image.


So what we try to solve is how do we generate tags that are more aligned with image such


that our pre training can benefit from this.


I think this this left picture here illustrates it well, where it just says from a bridge


near my house, right? Which is, which is a weird thing to put in an alt text, you would


put that usually in some sort of social media post or so. But this is one of the examples


where the alt text doesn't really describe the image. I thought I thought that was really


well, were you always aware of this weakness? Or? Like, how do you even find out that that


is a large scale problem?


Yeah, so I think I first come find out this problem when going through basically some


of the personal data set. So I think what people previously use a quite standard web


data set was this conceptual captions 3 million, which is a relatively medium scale is not


too small, but not very huge. And there are they do exist a lot of captions like this


in that data set. And I found this problem even exaggerates as I try to use a bigger


data set. For example, in this paper, we use the lion lion data set, which was a very newly


released data set. And the noisy problem was even more like, happens a lot more frequent,


when you try to scale up the data to include more web images with our text. So we feel


like this is something that if we can solve it, it could really change the models performance.


Have you seen the there's a recent paper called something like vision models are more robust


and fair when trained on uncurated data or something like this. So this here, you you


seem to say we need better quality data. And that group is saying essentially, no, our


models work better when we have less quality, but but you know, we just go out and collect


data. Can you maybe establish a bit of a connection between the two views? Like where do they


how do they agree?


Yeah, so I think maybe there's two different aspects. One is the quality at the other is


the diversity. So I think what that paper tried to maybe claim is I haven't read the


into detail is just my like, what my impression was that they try to claim if you have like


this huge web data set, that is more diverse, maybe then you're maybe human created data


set, you can bring better advantage to the model. I think that doesn't contradict with


what we say here. So actually, in our experiment, we show that the diversity of captions do


matter a lot. When we try to generate synthetic captions, we try to generate a diverse set


of captions that covers a whole bunch of different concepts, rather than a very common and safe


description of the image.


Yeah, I think maybe this two approaches that seem to me do not contradict but complementary


to each other. On one aspect, when you have more data, of course, you can always scale


up the size of your data sets, you are always having more samples that give you better capacity


for the model. But on the other side, we've more focused on the quality side, if you really


look at a number of images we're using here for the pre training, compared with some of


the other works is not a lot. It's not like too much to too large a scale. But since the


quality of our pre training corpus is better, we are now with a better performance. So I


really think the skill and the quality they are complementary and they do not contradict,


I believe.


Yeah. So let's stay on this on this pre sorry on the on the captioning and filtering. For


just one more second. You first did I get this right? You first pre you first pre train


the entire model on on this uncurated, let's say data set. And then you use fine tuning


on a human generated captioning data set in order to get these filter and captioning models


is. So my worry there would be a little bit exactly what we talked right now. What my


filter and captioning models learn is really dependent on, let's say, let's assume the


quality of the human generated data set is good. But the diversity of it really matters,


right? Because it sort of needs to cover all the images that come, you know, from the uncurated


data set. Otherwise, it is going to misjudge, misfilter or not being able to caption this


this data set. How do you you know, how do you control for that? And maybe you can also


comment on if I now let's say I want to expand my data set to areas that I know that the


human one doesn't cover, what could be a method of, you know, still going still going and


researching on this new type of data?


Yeah, I think that's a very good question. I think it's a valid concern that is fine


tuning, maybe bias the model so as to our certain domains. And I think one of the reason


we achieve performance improvement is because a lot of these downstream tasks are similar


to the cocoa domain image. So I think that's that's a valid point. But in the meantime,


I will say that this fine tuning doesn't destroy the model's capability to generate diverse


captions. And because the fine tuning is really a very lightweight procedure. So for pre training,


we portray on this huge data set for 220 epoch, which will take a few days, maybe a week,


but it's fine tuning, we only fine tune for five epoch, a very small scale cocoa data


set, which can finish within a few hours. So this fine tuning would not make the model


forget about what he has previously saw. It only slightly modify the model so that it


can generate captions that are more like human written ones. But we do find that even after


fine tuning the model can generate captions that are not within the vocabulary of cocoa


data set. So it's not like the fine tuning completely destroy the model's diversity capability.


So that's your answer to our first question. And for the second question, if someone wants


to try to expand the model to a different domain, where there doesn't exist human annotations,


I will say first, if you can collect some, it will be good. And if you cannot, maybe


one solution is there might be some similar images from this huge web data set that maybe


you can retrieve. So let's say if you can retrieve some similar images associated with


web captions, then maybe you can slightly fine tune the model on those subset so that


the model becomes slightly more biased towards your domain, and more suitable to your downstream


task. You suggest with this drawing, you suggest in with this arrow right here, almost you


suggest like a loop, like suggesting that this could be done multiple times, right?


I could, you know, go go multiple times through this stage. Is this? Is this anything? Okay,


I've, I've maybe not seen this in the experiment. If this anything you've tried, or would would


anything change in the loop number two or number three or number four? Like, what would


be the difference I have? I've already, you know, there's no new data introduced.


Yeah, so first of all, I would say it's definitely possible to do multiple rounds of iterations


of this bootstrapping. And in our future work, we mentioned this as one of the future work.


But in terms of extra knowledge, like each round of bootstrapping, we can add in new


captions, right? So if the model becomes better, it can generate better synthetic captions.


And there might be a diminishing return if we do multiple rounds. I would say my intuition


is the first round will probably help the most. And maybe the second or third will help


less. But unfortunately, due to the time and computation constraint, we didn't really have


the resource to produce the experiment before the paper. So that's definitely one of the


future plans that we have. Yeah.


So let's shift maybe, sorry. Okay. Okay, you. This model here is quite big. That's was my


first impression when I saw it. There's a lot of stuff. Okay, I have also drawn a lot


of stuff. Sorry, I can make this go away. So the model here is relatively big and relatively,


you know, there's there's modules going around, there's parameter sharing going on. What was


the what was the evolution of this model? Was this is this version one that we're looking


at right here? Or is this like, you know, version 50? After you've tried a bunch of


other things?


Yeah, yeah, definitely not version one. So actually, this model is heavily like inspired


by our previous LBF model, which is a encoder only model. So if you look at the model, there's


not too much difference between LBF and BLEEP, except the fact that now we add the generation


capability to BLEEP with the language modeling laws. So the reason why we want to add this


is first that because the encoder models doesn't really transfer that well to image captioning


task and other generation tasks. So it's better that we can portray it to have this capability.


That's why we add in this new decoder module. And then after we add in the decoder module,


we thought, since we are doing multitask learning, can we share some parameters? Because first


of all, it's more efficient to share parameters. And secondly, it may bring some advantage


from the multitask training by jointly optimizing those losses. So we tried different sharing


strategy strategy. First, we start with not sharing any parameters at all. And then we


try to decouple maybe the cross attention layer or the self attention layer or the feedforward


layer. Then we find that decoupling the self attention layer from the encoder and decoder


is a more efficient and effective way. So that's why we choose this strategy. But there


is a possibility that because we are doing this experiment on a relatively smaller scale


per training, so we're using the 40 million images for pre training, but our final model


was per trained on 100 million images. So maybe this sharing strategy is not the optimal


for if you scale up the data set. So I would imagine if you want to have the best possible


performance, you may want to scale up the data set and try to decouple the parameters


more. But that would of course sacrifice some of the efficiencies bring by the parameter


sharing.


Yeah, another point I probably want to add here is like this architecture is not like


ad hoc design, because remember that one of our starting point is to eliminate the noise


levels in this pre training data sets. So from there, on one side we need to identify


what are the noisy ones, whether the image and the caption they match with each other,


and that end up with this design of encoder model. On the other side, we want even more


that when we find that the caption does not align well with the image itself, we don't


want to simply discard training data point. We want to generate some useful captions,


surprising captions that can further help us. So from that, I really want to say that


it's not like we want to put everything together, glue different models into a single model


to make it big. It really serves very well for this caption filter algorithm. Yeah. And


I think that kind of, yeah. Yeah. Yeah. Just one additional comment is that our model is


really actually not big if you compare it to some other models. So basically our model


is a VIT plus a BERT. So it's a base version of the BERT. So in terms of the number of


parameters, I would say it's a standard parameter deep learning model. It's not that crazy huge.


So even we draw it in a current figure, actually there is because of this parameter sharing


going on, the number of parameters and the training computation load is not that heavy.


Yeah, I like the fact that this really arises from sort of the goal of cleaning the data


set in place. I also thought the more I read it, and the more I talked about it, it became


more evident that the things really play together nicely. You use the contrastive loss to get


the hard negatives for the, I want to say like matching loss or ranker loss. And then


that gives you the filter. And then the language model here gives you the captioning. With


respect to parameter sharing, you said, okay, the the matching head or the contrastive heads,


they're not really good at captioning themselves. So we'd rather pre train or train a captioning


or a language generation model. Do you find that adding the task of language generation


also helps the tasks that the other models would be good at? Like, do you find an additional


benefit except for our model can also do captioning? Do you find an additional benefit for the


already existing or the already tackled tasks by adding, let's say the language model?


Yes, yes, we find that there's an advantage brought by these language model loss. So this


language model loss, if you think about it, is really quite similar to the mass language


model loss, except that now it's the autoregressive version. So in our previous IELBF work and


some other papers, what people usually do is this mass language modeling to try to improve


the model's capability to understand the text in a more fine grained granularity because


the image text matching and image text contrastive learning is more like global matching, right?


You are trying to match the image and text, but the language model is more fine grained.


You want to generate the word based on the image and by achieving so you need to better


understand maybe some details of the image and align it with the textual concept to be


able to generate the word.


Do you have, let's say, more extensive goals in mind here? You just said it's actually


not that big, you know, if it's really nice, I agree with all of that. Yet I foresee a


future where you could, you know, bring together lots of these modules. Essentially, what I


what I'd like to have is, first of all, we could obviously think of doing the same with


the image side right here, you just have an encoder here right now. But we could think


of, you know, breaking out here doing image generation doing, you know, what whatever


we can do with images. But on the other hand, maybe an even bigger future vision would be,


I bring a data set and I say, Look, these are pairs of images and text. Now, please,


system make me a model that includes all of these losses that I can think of, like all


of these different combinations and the system would figure out, okay, I can share, you know,


I can share parameters here, and I can build that and so on. And maybe that would, given


your findings, which I, you know, I totally believe that adding more of these tasks and


sharing the parameters actually mutually benefits each other, the representations, they become


more capable, they become maybe more more broadly meaningful, and so on. So I think


that might be a cool, a cool future to to work against. I don't know how feasible it


is, though. Is that anything on your roadmap? Or, you know, what does the future look like


of these models?


Yeah, I think that's a very cool idea. Maybe very ambitious goal. So we have considered


to ID in some image generation capability, but we didn't because it doesn't fit very


well with our current framework. So we don't want to make the framework to be very huge


and messy. We try to keep it more cleaner. But regarding your point that can we have


automatic system that can maybe combine different modules and losses? I think that's a possible


goal. It's just there could be a lot of obstacles in how to achieve that. For example, if we


borrow some idea from the mass community, and maybe we borrow some reinforcement learning


idea, maybe there are some ways we can train a policy to do that. But it's not entirely


clear to me how how can we achieve that? Because I think the main problem is this per training


is how to evaluate a per training is a big problem, right? So you cannot just say that


the lower per training loss means that your model is better downstream task. If there's


a correlation between per training loss and downstream task, then it may be easier, right?


You just find the optimal module that you can minimize your per training loss. But usually


it's not the case. It also depends on how well aligned is your per training task and


your downstream task. I think that's one of the major issues of why it may take some trial


and error to find the best strategy for the per training.


Maybe I can add a few sentences to that. I think being able to figure out how to combine


these different modules together automatically would be super cool and futuristic. Yet I


think there are a couple of practical messages that we want to convey here, which is the


first I think if you really look at how this we fine tune, we fine tune this MED model


to make them a captioner, a filter, and also how we combine these different modules together


in order to tackle the downstream tasks. There are really some dedicated ways to do that.


And usually if you look at some pre training works on the market, their strategies will


be pretty simplistic in the sense that in most of the cases they just add the task specific


heads. But in this particular work, we just move one step further than that. We are rethinking


how to rearrange these modules and what are the best strategies for this parameter sharing


strategy. Another message we may want to say here is a lot of people, they blindly do this


multitasking by aggregating hundreds of different data sets and tasks into one pre training


model. And maybe by BLEEP we want people to kind of revisit this decision next time they


do this multitasking because not necessarily every task they complement with each other.


And you may want to carefully look into what to share, what not to share. I think these


are the two things we want to remind for future works.


And I have one additional comment to follow what Dongxu said is that you can see a lot


of other works, they really combine maybe eight or ten objectives together. So there


are some strategies for vision language training is you bring in an object detection objective


to improve your localization capability. So we think that's a valid way to improve performance.


But here what we try to say is that we want to keep things very nice and simple. So we


have these three laws where each law serves a very clear purpose and can be transferred


to a very specific downstream task. And all we need is just image taxpayers. We don't


need any bounding box or anything else. So I think that's what the message we want to


also convey.


Cool. And yeah, and I especially I like the fact that with pre training with the aspect


of fine tuning, then you're able to recombine these different modules in in very creative


ways. So even even though you have these modules, they have their purposes for the pre training


for the captioning for the filtering, but then they can be it seems it seems many, many


tasks can now be tackled by some sort of combination of these models and a little bit of fine tuning,


which is something that I find really cool. You have done extensive and like, there are


there are lots of lots of tables means means you had to run like and collect lots of numbers,


which is, is very nice, because gives a bit also of a broad overview than just having,


you know, four numbers or so comparing with one baseline. Although could you maybe highlight


some of the of the standing out results that you got? Or one of some of the more important


results? Like how would you summarize or what would you highlight about your experimental


evaluation of this?


Yeah, sure. I think the most important one would be table one, where we demonstrate the


performance gain achieved by how do we bootstrap our data set? Yeah, and yeah, so this is table


basically, if you look at the first column, it shows how many images we're using. So we


have two settings, one is a 40 million images. Another we scale up with more noisy image


taxpayers. And the second column is how do we perform the bootstrapping? C stands for


captioning and F stands for filtering. It means whether we do captioning to generate


synthetic captions, or we do filtering to remove the noisy captions, or we do both together.


So if you look at the first row, second row, third and the fourth row, you can see that


both the captioning and the filtering can help individually. And if you combine them


together, they really have complement each other. So by generating synthetic captions,


and at the same time, try to remove the noise, we can achieve, I would say a quite good amount


of gain in these four different data sets, covering both the retrieval task and the captioning


task. So I think that's one of the key results we have here. And also maybe then it goes


to the second table is how do we do the bootstrapping of the captions, right? So do we use beam search,


or do we use nuclear sampling? So the difference between those two approaches that beam search


is a deterministic sampling, not sampling, deterministic decoding strategy, where you


try to find the most likely sentence associated with the image. And nuclear sampling is a


stochastic approach where you try to sample according to some probability distribution.


And we find that surprisingly, if you compare beam search with no generation, there is a


good gain achieved by beam search. But by moving beam search to nuclear sampling, there


is a similar amount of gain. So this is something that we didn't expect at the first time we


see the results. And after we really deep dive into what the captions look like, how


does beam search and nuclear sampling generate different captions, we found out that the


beam search will generate a kind of a safe caption that accurately describes the image


most of the time. But it's not surprising. So you can commonly see those captions in


the data set. And that doesn't add a lot of extra knowledge for the model to learn. But


the nuclear sampling really introduced some really diverse captions that are more like


human written ones. The human don't write a very boring description like a man is with


a dog in a park. So it's a very boring question, boring caption. But nuclear sampling can give


you more diverse captions. And if you look at a noise ratio, which is actually how much


of those captions were filtered out by our filter, you can also see that beam search


is less noisy. But even though it's less noisy, it's not as beneficial as nuclear sampling


here. And this really raises another question, which I think is a very interesting future


work is that is nuclear sampling the best way, right? So because those models are pertrained


with the language modeling laws, which is kind of deterministic laws, you try to maximize


the likelihood of your captions. And we are just doing that. And we try to do something


in the decoding side to try to give more diverse captions. But this nuclear sampling was used


in mostly NLP papers. So is there exists some better diverse captioning strategy for image


captioning task? So I think that's a very interesting question.


I think in recent times, this has been shining through in a lot of works, that the fact that


maybe we don't need to go maximum likelihood in our inference step, but maybe it's a better


approach to go diverse with the sampling. And then exactly what you do have some sort


of a classifier or some sort of a filter to just it to just scrap out the noise. I think


that's a really, really good approach. And we saw this, you know, anywhere, I think,


Dolly famously had had clip re ranking all the outputs. And I think more and more models


go towards this. It's really cool, really cool finding that you're essentially you're


finding exactly the same thing. When I look at these numbers, all of the numbers is very,


let's say, is very convincing to see that everything uniformly almost almost uniformly


gets better, right? You know, you're you support whatever you say really well. I mean, this,


this trend right here, it's it really works across, let's say, across all of the data


sets, you uniformly almost get better in all the tables. However, the difference is always,


you know, there is the maximum differences, whatever this this from here to here is like


two points in what what is this? What's tr it's true. It's a recall, text recall, text


text recall, sorry. Oh, yeah, it's down here. Okay. Text recall, image recall. That's like


2%. Right here. Again, it's like one point something percent. So there's a uniformly


getting better. My question is, given that the getting better is convincing, but the


scale of it is like, yeah, 2% or so. When is it worth to do this weeks long or week


long pre training you mentioned, right? This is a big procedure, the pre training is big,


and then the fine tuning and pre training again. When is it worth it? From what scale


or for what applications does it become actually worth to do something like this?


Yeah, I think that's a very good question. And first of all, I would say it is worth


doing if your data is really, if you observe a large amount of noise in the data, and maybe


your data is incomplete in some of the domains. For example, here, the web data is primarily


dominated by those all texts, which can be different from what human would write to describe


an image. And so there if there is a noisy scenario or domain gap, I think it's worth


to do so. And secondly, actually, we have also released our data set after bootstrapping.


So that if you are just trying to do regionally protruding in a similar domain, I think you


can just download our version and use that as a starting point to avoid the first round


of pre training. And maybe certainly, about your previous comment that we have a really


unanimous improvement for those tasks. Actually, in one of the tasks, maybe you can scroll


down the paper. Let me try to find I think it's what the NLVR task. Table eight, maybe.


Yeah, yeah, table eight. Yeah, actually, for this task, right, this is where we find the


better quality of captions doesn't necessarily give you a better game. If you compare here,


and actually by scaling up the number of portraying image, it doesn't correlate very straightforwardly


to a downstream performance game. So I think it still depends on your alignment between


our pre training and your downstream objective. So for most of tasks, it is well aligned. And


that's why improving our pertinent data quality can improve your downstream tasks.


Yeah, maybe I can add a few sentences to in terms of why there it is worthwhile to improve


that much. I think if you really imagine the big picture here, in terms of the multi model


retrieval, let's say, if you deploy this retrieval, and that managed to improve their profit by


1%, that's a huge achievement. You want a lot. So at Salesforce, we also have the retrieval,


we have we also work with clients for their retrieval services. So in terms of that, if


you just let your GPU run for one week and improve by 1%, that's a huge improvement,


I would say. And I also like to say that these numbers, they kind of, I think, under hype,


what BLEEP has achieved, because I think BLEEP beyond this relative advantage over its competitors,


is also qualitatively better in terms of how easy it is to use BLEEP. If you really look


at the demo we created there on the web, hosted on the web, and it just freely asks any questions


in natural language rather easily. In contrast, a lot of this image question answering models,


they are kind of they're not doing the freeform generation, right, they're kind of doing classification


in order to tackle this question answering task. This one is however, not fully demonstrated,


I believe, in the current manuscript. So if you really want to get impressed, we really


suggest you check out our demo and put whatever photos you like and questions.


Cool. It's really neat, by the way, that you have like a demo to go along with it. Because


I think it makes it makes it more accessible. And it demonstrates also the capabilities


of this, it's almost like we're moving into it, it's, it's we're moving into the world


that GPT-3 maybe has created for text with these image language models. Because, you


know, we got the same feeling from GPT-3. Oh, no, you can I can just go and I can put


any text right and I can interact with the system in a sort of a freeform way. And it's


really cool to see that we're also moving in this direction with with the image models.


In in terms of in terms of just the process of how this is research went about it, you


ended up with a cool system with a nice way of bootstrapping data and so on. Was there,


can you maybe tell us a little bit about stuff that didn't necessarily work out during the


research? Was there any point where you were maybe disheartened a little bit things that


didn't work out? What were your low and your high points during this the creation of this


paper?


Yeah, actually, one of the like the experiment we had was when we first tried to scale up


the Petrino with more web images using this line data set that we have downloaded, which


takes quite some time. It doesn't help that much. So then it feels really feel like why


scaling up the data is not benefiting the model. So then I did some more analysis. And


after that, I realized that a lot of those images are very, very small in the resolution.


Some are just icons or some brand names. And if I remove those, then it begins to show


the gains. But I think that's one of the kind of the blockers we faced. And I think after


we first get the bootstrapping, especially the nuclear sampling, to give a big performance


gain, then at that point, we are quite confident that this should be a good solution. And I


think that point is when I realized, okay, this method should work well, and we can write


a paper about it.


Good. Dongxu, do you want to say something?


Yeah, I believe some of these strategies, they also arise from the internal discussions


with other group members at Salesforce. So it's really a lot of crowd intelligence behind


the scenes. So yeah, that's


How is research organized at Salesforce? Like I have a bit of insight into, you know, the,


let's say the big tech giants like Google and Facebook and so on. And they have their


research divisions at a company like Salesforce, who is more customer, I want to say customer,


all these companies are customer oriented, obviously. How is research organized there?


Like what do you do while the model is pre training for a week? Like do you have do you


have other stuff to do? Or are you mainly researchers? Or what's life like there?


Yeah. So first of all, I would say that AI is a big part of Salesforce, what they try


to achieve, like to use AI to better help the customers. So we have this separate research


division, maybe not as large as Google or Facebook. But I think everything works quite


well in our research team. And in terms of our day to day operation, I think it's mostly


similar to other industrial researchers, we, we can quite flexible to do research or do


some more product oriented work. And, and like, we are motivated to do research that


can generate high impact, that can really change the field in a more substantial way.


And while we wait for the GPU to finish training, you already would just do other research stuff


or read some papers involving some internal discussions, or maybe try to solve some real


production problems.


Cool. Is there anything else you want to get out about this paper? You already said people


can go to to the web, to your repo, and you have a you have a demo also available. Is


there anything you'd want to get out? Like how can how how's what's the easiest for people


to get started with this research?


Yes, so I think first, again, welcome to try out our demo and welcome to visit our GitHub.


We do have, I think quite detailed instructions on how to download and portray or find you


on the model. And also I welcome any suggestions or questions you might have about our model


that we can use that to improve our model or the code. That would be great.


Dongxi, anything, any last messages?


Our team is expanding. So if you are interested, just let us know.


Yeah, we are looking for intern position in the vision language research.


Cool. Who can apply? Anyone that is at university or?


Yeah, anyone can apply. We hire globally so we can do remote working now.


Cool. Excellent. Okay, Dongxi and Jinan, thank you very much for being here. This was a lot


of fun.


Thank you for having us.


Symbolic Knowledge Distillation: from General Language Models to Commonsense Models (Explained)

Hi there. Today we'll look at Symbolic Knowledge Distillation from


General Language Models to Common Sense Models by Peter West and


others of the University of Washington and


the Allen Institute for Artificial Intelligence.


On a high level, this paper takes a new approach to symbolic knowledge generation.


So to automatically coming up with knowledge graphs,


with symbolic knowledge graphs.


And rather than trying to mine this symbolic knowledge automatically


from raw text or from existing knowledge bases,


they mine it from GPT-3.


So they use the GPT-3 large language model in order to first come up with a corpus


that gives them a corpus of symbolic knowledge.


And then they use that corpus in order to train a model that they call a common sense model.


But essentially a knowledge graph completion model.


So this is a new paradigm where you go,


what they say, from machine to corpus to machine.


And it is the paradigm they advertise here,


in contrast to what people did before,


the from human to corpus to machine,


which is where humans generate a corpus,


and then you train the machine on that corpus.


So we're going to look into how they do it.


It's pretty surprising what they find in that, for example,


the distilled model, the models they come up with at the end,


they tend to be better not only than the humans or the human fed models,


they even tend to be better than the original teacher, the GPT-3 teacher.


And this is a result of how they combine the different elements here of the system.


And they strategically bring in outside help in the form of human knowledge.


So this could be a recipe for much more broad applications,


not only knowledge, knowledge graph generation,


but various natural language tasks.


They combine cleverly prompting, training small models,


and as I said, bringing in small amounts of human annotated data strategically.


So, as I said, we'll go through it, we'll look at the different stages.


And yeah, tell me what you think in the comments.


Subscribe if you haven't. And let's dive in.


But first, a quick word from our sponsor, Weights and Biases,


your one stop shop if you're a machine learning researcher, practitioner,


a hobbyist, a power user, it does not matter.


Weights and Biases is with you from the inception of your idea,


tracking your experiments to really getting the fine details right,


optimizing your hyper parameters up until you deploy your model


and track all of your metrics.


Not only does it do that, it also organizes your data sets, your models,


and you can generate super cool reports from all of that.


In addition to that, it lets you have great insight into what you research


and what you produce.


And all of this runs in the cloud really effortless with a single line of code.


Though today I want to talk to you about a yet not so well known feature of Weights and Biases.


And that is the Weights and Biases community.


So I believe they recently migrated this from like a giant slack


onto this new sleek community website.


It's a discourse based forum essentially where you can get help not only for Weights and Biases stuff,


but also machine learning in general.


But not only is it a help page, it's a discussion forum about all things machine learning.


Also, they organize regular events, book reading groups and paper discussions and so on.


So if you're interested, don't hesitate and hop over to the introduce yourself thread


and take part in the discussion.


As I said, this is still a pretty young place, but it's bound to grow over the near future.


And of course, if you want any advice on Weights and Biases, how to use it,


what are the best practices are, this is the best place to do so.


Thanks again to Weights and Biases for sponsoring this video.


It's an awesome system.


I invite you to check it out and back to the video.


So what's the deal with knowledge?


I can't read this without pronouncing knowledge as knowledge.


So what you want to do is you want to have symbolic knowledge.


And in this particular case, the symbolic knowledge they're after is what they always have to have


what they call an event and a relation.


So an event, relation, an event, they give some examples.


But essentially, the event is some kind of situation that a person finds themselves in.


It's common sense reasoning.


So it's not like Napoleon was born in France or something like that.


I don't even know if that's true, but it's not that it's common sense reasoning.


So the event is a person finds themselves in some sort of situation or two people, it


can can be one or two people, then the relation is some sort of, well, it's probably better


we make an example, the relation is some sort of this, for example, this is the situation


right here, x starts running.


The relation is these are predefined relations, and we deal with seven different relations


right here.


The seven relations are chosen because they represent sort of causal, causal knowledge.


One of them is effect, which means what is the effect of this event?


Or what is one possible effect of this event?


And the goal of the model is to come up with this thing down here.


So you prompt the model by saying x starts running, we have the effect relation.


So the model is supposed to come up with the effect of starting to run.


Now there's not only one correct example, there are many correct examples right here.


But one example is x gets in shape.


This is not a direct logical, you can't prove it mathematically, right?


Or you can't check it.


And that's why it's called common sense reasoning.


A human would look at this says x starts running is the effect of that that x might get in


shape?


Yes, probably.


So that is a valid triple.


Okay, let's look at another one.


Let's maybe take one with two people in it.


No, there is none with two people right here.


Let's see.


X is not well liked.


That is the event, the relation that we give to the model right here is the react relation,


which means how how does a how does x react to that event?


So x feels lonely.


And that as well kind of makes sense, right?


If you you as a human judge this, you apply your common sense makes sense.


So I hope the task is clear given an event and a relation where the event can be any


anything like any thing involving x or x and y, which are one or two people, and any piece


of text, right?


This is any piece of text right here.


And a relation the relation they are seven different predefined relations.


You have to give the result right here the inference and the inference again can be any


text.


So this is quite a challenging task, right?


And humans have come up with a data set for this task.


I don't know where they describe it right here.


They have come up with a data set called atomic 2020.


So the atomic data set is a data set that where humans go and humans make these triples,


right?


And it's a data set made by humans, as you would make data sets, this takes a lot of


work costs a lot of money.


And we would like to have methods for not having to do that necessarily.


So either to cut out the humans altogether, or to use the human labor more strategically,


such that it doesn't cost as much.


And they also the model that's trained on this human corpus, it's called common, sorry,


comet 2020.


That is, if we simply feed the human corpus to a deep learning model, have it learn to


predict the inference from the event and relation, that model is called comet 2020.


And that's going to be our baseline.


And obviously, we're going to surpass that.


So the result of this paper is going to be a another corpus called atomic 10x, which


is 10 times the size of the human atomic data set, which is going to be better or larger.


And with appropriate filtering also better in quality than the original corpus, which


is surprising, right.


And then also the comet distill model, which is the model that's trained on the atomic


10x data set.


And that is going to be as well, depending on the filtering, largely better than the


original comet 2020 model that's trained on human data.


So that's the goal that we we get there, we get to a model that is better than it had


we trained on human data.


And along we get a corpus that we that is better than the human corpus.


So again, the original, the original paradigm was humans go humans think with their brains,


like you're from the brain comes a corpus, right.


So I invent a bunch of corpus entries, right?


Maybe I'm many like many, I let many humans do this, I come up with a corpus manually,


and I feed that corpus to the model through the machine.


So there is a neural network right here.


I trained the neural network on that machine neural network thinks, yeah, cool.


The new paradigm is the following.


I take a big giant neural network, such as GPT-3, that is not necessarily trained on


this task, right?


I'm going to make GPT-3 have one more layer than the other network to symbolize its absolute


bigness.


So GPT-3 is trained on the whole world wide.


Is this a globe?


This is a globe.


GPT-3 is trained on the whole world wide web, or at least readable part of it.


And I'm going to use GPT-3 in order to come up with the corpus.


So I'm going to use GPT-3 to come up with this corpus.


And then optionally, optionally, I'm going to filter that corpus with a model that I


train on human data.


So this is where the human component can come in right here.


Now, we're going to see how this happens, but the obvious, the obvious effect of this


is that the human no longer needs to come up with examples, the human simply has to


rate examples in order for the filtering mechanism to get better, which is much easier, much


cheaper.


And we don't need as much, I guess, maybe we do.


But it's, it's essentially, it's much cheaper for the human to rate than to come up with


stuff.


So we use GPT-3 to come up with a corpus.


And then we use that corpus to train our model.


So we're going to use the power of these large language models to come up with corpus.


And of course, the magic is going to be how are we going to do this?


And the answer is clever prompting.


So there's a bunch of math right here about knowledge distillation.


I'm not sure.


I guess they just had to put this in to get accepted because you need like a bunch of


math and yada, yada, yada.


But essentially, it's irrelevant.


So yeah, sorry, if if you disagree authors, but yeah, this is, it's essentially irrelevant.


So the key findings of the paper, actually we're going to skip this, because we get this


at the end.


So what do we mean by clever prompting?


We want to come up with a corpus.


The corpus should have events, the corpus should have inference relations, the relations,


of course, we know the corpus should have inferences.


So they have this general template for prompting GPT-3.


They start off with a task prompt where you briefly describe the task inside the prompt.


And then they have a bunch of examples.


So the input, the output, the input, the output, the input, the output, and then they have


another input.


And this is the input they're actually interested in.


And they're going to let GPT-3 complete the output right here.


Now given that they have the task description right here, and they have this pattern of


repeating inputs and outputs, you can get GPT-3 to continue the pattern and actually


give you what you want right here.


We've seen this a number of times right here, this is called prompting or prompt engineering.


And I predicted this right away when GPT-3 came out that prompt engineering would sort


of be like, it's a quite an important thing to do in the future.


So importantly, we don't train GPT-3, we simply query GPT-3 in a very structured way in order


for us to create a data set essentially, I think that's even against the terms of service


of GPT-3.


But they must have gotten an exception here.


This paper is also cool because it finds a number of interesting things in prompting


that some of you might have been aware of this other is not but there are interesting


effects.


For example, you want to number these things right here, you want to label them with actual


numbers such as that.


They say this increases the degree to which GPT-3 follows previous examples.


And also, when they construct examples, for example, like this X goes jogging, they also


say if they replace X and Y and so on, by common names, it also works better.


So you really want to, I think it's, it's still a bit of an art form to see exactly


how you have to phrase the things you put into GPT-3 such that you get out something


good.


So the first task they're going to do is they're going to create these events.


Ultimately, we want to create the data set.


But the first step is we create the events.


So they go to the atomic data set, this human generated data set.


And what they do is they simply sample, so they collect a set of 100 high quality events


from atomic 2020 to use in our prompt.


Note that yes, they do make use of the human corpus right here, which is a little bit unfair


when you think of comparing to that.


But given that it is 100 examples, that is something you could still easily come up with


even even as a researcher, right?


Or you could you could pay a bunch of humans 100 examples isn't that much.


So we go and we collect 100.


And then we simply every time we go to GPT-3, we randomly sample 10, we put the 10 inside


of the prompt, right, we simply list the 10 events.


For example, x overcomes evil with good, x does not learn from y, and so on.


We simply list that and then we put 11 and we let GPT-3 continue the prompt right here.


And that here is going to give us an next event.


I guess we could even let it continue more.


But there are these issues like repeating and so on.


So I'm not exactly sure how well that would go.


But in any case, you can generate essentially infinity events.


Because even if you even if you put the exact 10 same events in the exact same order, right,


since you sample, you sample with with nucleus sampling, it doesn't give you the same results.


Therefore you can generate a lot of events.


In fact, they generate 165,000 unique events, which is, as you can see quite a bit more


than the human authored corpus, which only has 6.2 thousand events.


And all you needed as a base is 100 of these events, right?


100 were enough in order to create 165,000.


That is the power of these large language models.


You can essentially count on them already having built in all of this sort of language


sampling, all of this, well, you might call it knowledge, or you might simply call it


data that they have absorbed.


But you can query that in a particular way.


And the way we query it here, it gives us new events.


Alright, so this is the way pretty simple, that we create new events.


Now from these events, we want to create these triples, right, the triples are going to


actually make up the data set.


So for a triple, remember, we need an event, we need a relation, and then we need an inference.


So the events we now have check the relations, there are just seven of them, they're always


the same in this data set.


So we have them as well.


So now we can simply pair, take an event from the data we created, pair it with a relation,


and then we have to come up with an inference.


And again, we're going to use clever prompting, and GPT-3.


So what the authors do is that for each relation, they come up with a, they come up with a textual


representation of that relation.


So the by the way, the the relations are described right here, there is x adder, how x is perceived


after an event, how x reacts in response to an event, what effect does it have on x, what


was x's intent in event, and so on.


So these are the kinds of relations that we're dealing with right here.


They give an example here for the need relation, which is here, what x needed for the event


to happen.


And their textual representation is as follows.


So I'm going to put the event with an event number right here, according to what they


said at the beginning, it helps when you number the individual entries, then they're going


to write prerequisites for this to happen, comma, and then the actual inference goes


here, right until here.


So they're going to repeat this, this is one, they're going to repeat it two, three, and


so on.


And then they're going to put 10 samples into the prompt with the inference filled out.


And then for the 11th one, they're simply going to put the event right here, and the


prompt that they have already used.


And then they're going to let GPT-3 fill in the rest right here.


And that thing is going to be the GPT-3 provided inference.


So they say, as in 3.2, we sample 10 few shot examples for each prompt from a set of 100


human authored cases.


For each pair of event and relation, we generate 10 inferences with the second largest form,


following the same hyperparameters as event generation.


Now they don't use the largest form of GPT-3 because it will cost them too much money.


So they use the second largest one.


But you do the same thing, you you generate just very, very, very few human authored cases.


So that's 100, 100 human authored cases.


And I don't know if that is 100 per relation, or just 100 in total.


I don't know.


I'm going to guess maybe per relations.


It doesn't say, just says we replace anonymous names with generic names as this improves


quality.


However, it doesn't matter if it's 100 or 700.


It's still very, very few compared to having humans come up with an entire corpus.


So what you want to do is you simply want to give GPT-3 a little bit of input, like


10 different things of input.


And these 10 things, you may vary a little bit over time, you might not even have to.


And let's not forget the task description up here, that also seems to be important.


And then they come up with 165,000 times seven inferences, which you can filter a little


bit.


But in the end, this results in 6.46 million atomic data, atomic style data triples.


They call it atomic 10x, as it contains an order of magnitude more triples than the atomic


2020 with respect to the seven relations they investigate.


So this is a giant corpus right now of machine generated, of machine generated data.


I'm trying to find table one where they compare the size right here.


Okay, so here you can see just the comparison of what that cost, you can see the total count


in atomic 2020 is 600,000 triples, and atomic 10x has 10 times more triples, yet cost only


a fraction of what atomic 2020 cost.


Now the question is, of course, is this data set any good?


This here at least has been generated by humans, you know, humans aren't perfect, but at least


they have some common sense, therefore, for a common sense data set, it might be important.


Does the atomic 10x data set, is it any good?


And that's what they go about investigating right now.


So they evaluate the generated common sense knowledge graph.


So they evaluate now these triples.


First of all, they look for diversity.


So they have a few diversity related metrics, such as like hard diversity, or this what


they call blue soft uniqueness, where they check for overlap between the triples and


look how many of them are unique.


They also look, they also try to train a GPT-2 model and look at the entropy of the different


data sets.


And in general, they find that the machine generated data is quite diverse, has quite


high entropy, there's not much of a problem right there.


It's also quite unique.


It is not as unique, it seems, as the human generated data.


But given that you have so much more of it, the absolute number of unique things is way,


way higher.


The real kicker comes when you do actual human evaluation.


So they've spent a lot of time into humanly evaluating the quality of whatever they produce.


The humans have been asked to rate these triples into, for example, always often.


So when you see an event, a relation and an inference, you as a human have to say, does


this inference always or often come from the event and relation?


Is it sometimes?


Is it likely?


If you said one of the two, it would be accepted, the triplet would be counted as good.


If you as a human say, ah, that's kind of far fetched, or that never happens, or is


invalid, then you would reject the triple.


If you look at this, then you can see right here, in the human authored data set, the


humans accepted 68% of the triples and rejected 11%.


Whereas this top row right here is the unfiltered data set we got from GPT-3 with the prompting.


And you can see that the accept probability is slightly lower, actually quite a bit lower,


like 8% lower.


And humans also reject more often, and even sometimes not available means that you can't


make any any judgment on it.


So the number is, it's way larger, right, but it's a bit lowering quality as assessed


by humans, as it seems.


So now they gear up.


They say, okay, can we make this better?


And their answer is yes, by introducing a critic.


So making the teacher model more critical, where they go about the following, they have


this formula right here, maybe that math isn't as useless.


After all.


So if you simply generate language, you simply have GPT-3 be a model, a probabilistic sequence


model, a language model that simply says, what is the probability of the next token?


And I'm going to sample by that probability.


But now what you can do is you can introduce a critic.


So if this is your language model, you can introduce a critic.


And the critic also will have an opinion on how likely a particular sequence is.


So now you consider both you can you generate data with GPT-3.


And then you let a critic evaluate that data, which essentially amounts to multiplying the


two probabilities.


But in practice, you would simply run the critic on the data.


And then the critic decides, is this data good data or bad data and that together GPT-3


and the critic, they you hope that they will produce a better data set than just GPT-3


alone, because now the critic is able to filter whatever GPT-3 says, and only let the good


data pass.


Note that I think it's maybe the critic is probably capped at one or something like this.


So this is a filtering mechanism.


It's not like you can, you can introduce new bad data.


So we would expect that the filtered corpus is hopefully better.


The question is, how much better is it?


Okay, so now we introduce this critic.


And the critic is now is where we strategically bring in human data.


The critic would remove unacceptable knowledge in practice, this means filtering the generations


in the large corpus and creating a range of new corpora that are higher quality yet still


larger scale than the human, the human authored one.


So for this, they gather a training set of correct versus incorrect humans, human judgments


on a randomly sampled set of 10k entries of atomic 10x.


So they take their large corpus, they take 10,000 entries of it, and they let humans


rate those 10,000 entries much like they did here for the evaluation.


But this now counts as this now goes as training data for the critic.


And that's where I said we strategically bring in human knowledge.


And not only do we strategically bring it in rather than letting humans generate the


entire corpus, we also make it easier for humans because this isn't coming up with examples,


coming up with examples is hard, it takes time.


These humans here, they simply need to read examples of the corpus, these 10,000 examples,


and for each one, they have to rate it.


And this can even be noisy.


So other than in the evaluation, where I think they gather three labels per data set, they


say we only gather one annotation for each example.


So this can be noisy since it's training data.


And yeah, that seems to be quite a quite a good way of thinking about human labor in


machine learning.


It's sort of where can we bring it in to make the biggest difference?


Now when they do that, yeah, so they argue this here, it's vastly cheaper than human


construction.


Instead we argue that a more useful and efficient role for humans in knowledge graph construction


is to correct the mistakes of the teacher by evaluating a small number of examples.


So they train a Roberta large model on the human annotated data as the critic, the critic,


of course, doesn't have to be a language model, it doesn't have to generate anything, it simply


has to look at the data and decide is it good or is it not good.


So they train that and, and, and yeah, now we go back to the table right here.


These here, as we go down the table, more and more filtering is applied by the critic.


So now you have a choice as a designer, right?


You have this critic model, it tells you about how good a particular sample is.


And now you get to decide the cutoff, you know, how much do I want to filter this data


right here.


Now, this will have a trade off, the more you filter, the smaller the resulting data


set is going to get.


So we can look at a few examples.


For the first step, you go from 5.6 million, sorry, from 6.5 to 5.1, which is a reduction


in somewhere between somewhere on the order of 20% of data.


So you throw away 20% of data, look at that the accept percentage jumps from 78% to 88%.


So now human raters, human raters rate these triples in the corpus that you generate and


then filter as more likely, more acceptable than the corpus that was authored by humans.


Like this is, this is astounding already, right?


Now there might be a little bit of an effect here in that probably the humans that rated


were the same humans or at least, you know, humans from the same population or distribution,


then the humans that rated the training data for the critic, and therefore, all of these


humans might sort of have the same taste, whereas the humans that came up with the atomic


2020 data set might be different humans.


I'm not sure, but it is astounding.


And even more astounding, as you filter more, you can clearly see the accept percentage,


therefore the quality of the data set going up.


And to the point where you keep about 40% of the data that you've generated from GPT-3,


that the accept percentage is like 96%, which is 10% higher, 10 percentage points higher


than the accept percentage of the human generated data, right?


This is quite, this is quite astounding.


And still, you have like four to five times more data than the human created corpus.


And they do some, they do some, they do some evaluation also, again, on the diversity of


the data.


And actually turns out that as you go, as you filter more, the diversity increases.


So that would be the relative diversity, meaning sort of how many percent of the data are,


you know, different from other, how unique and so on.


So it appears to be that GPT-3, when it just creates data, it will create a lot of good


stuff, but also some garbage.


And as it turns out, the garbage seems to be always the same kind of garbage.


Therefore, if you filter out the garbage, also the uniqueness and diversity of your


overall data set increases.


So it's quite the opposite of, you always hear this, no, I guess, I guess it's that


the saying that all, was it all unhealthy families are the same or all healthy ones?


I don't know.


But in this case, all the garbage GPT-3 produces is kind of the same kind of garbage or the


same few types of garbage, whereas all the good stuff it produces is relatively unique.


All right, so now we have a really, yeah, this is what gets filtered out right here.


So first of all, logical misalignment consists of events or inferences joined in a logically


inconsistent manner.


That makes sense that that gets filtered out.


X cannot find his shirt.


As a result, X is wearing a shirt.


Yeah, that should probably not be in there.


And two awkward phrasings which consists of events or inferences that in isolation are


incoherent, ambiguous or awkwardly phrased.


So when an event itself is already poorly phrased, the model essentially has no chance


of generating good inference like person X has a fire in the bath.


Yeah, so there is just, there is a high chance that a human would negatively rate this or


not accept it or say it not available.


Like from the get go, doesn't even matter what the relation and the inference is, right?


So the last step is, the last step is we want to go back to a model.


So we have taken GPT-3, a model, we have used it strategically to come up with a corpus


that is both better in quality, more diverse and larger than the corpus that humans have


generated.


And now we want to go back to creating a model from that corpus.


So I want to train an inference model because right now we can only generate data.


But we would like to have an inference model.


And remember, the original task, the inference is to given an event and a relation to produce


and to produce either produce an inference, right, which you could do with GPT-3, but


it's it's sort of not super good.


So you have to filter with the critic, but that means you have to like sample until the


critic says it's okay.


What you'd rather have is you'd like to have a model that is trained on this data to produce


directly the inference rather than having to prompt GPT-3, right?


So the model can be way smaller than GPT-3 because it's directly trained on the task.


And you don't have to pay OpenAI every time you call it.


So now I want to go back to a model.


And that's pretty easy, right?


We simply take a the same architecture as this comment model.


Remember the comment model is the model that's trained on this human data to do this inference,


simply take same architecture, and we train it on the large corpus.


And what what turns out so on?


It turns out that we do that, and then we let again humans rate the triples that the


models produce.


So for the Comet 2020, this is the model that's trained on the human corpus.


This here you can again see the accept percentage by the raters of of the corpus itself.


When we train the model on it to do this inference for us, the model produces triples that get


accepted 81% of the time, which is pretty good, right?


So if the corpus gets accepted this much, we train a model on it an NLP model.


It's pretty good to drop only a little bit in the accept percentage.


That means the model has essentially learned because this is obviously on a on a validation


set.


The model has obviously learned to do this inference somewhat correctly.


Now if we do the same on our large corpus that has lower accept percentage, we see the


same effect.


So the model kind of learns in fact, overall, we see the same effects.


If we now add a critic with a low threshold, then we surpass already this model.


And we if we add a critic with the high threshold, so that would correspond to throwing away


60% of the data as we saw before, then the model that we end up with has an 87.5% accept


rating.


So now we have a model that's the same size as this comet 2020, right?


It is an a trained model.


It's not GPT-3.


It's not prompting.


It's a trained model that does inference in these triples.


And it is better.


It is better than the model, the same model that's been trained on the human corpus, which


is pretty cool, right?


So you even you it not only does it surpass GPT-3 itself, it also surpasses the human


generated data.


And yeah, that's pretty cool.


So this was essentially the findings of this paper, I guess we can go back to conclude


with what they said at the beginning, the key findings right here, learning symbolic


knowledge from language models can be framed as a symbolic extension to knowledge distillation.


Okay, so that's the that's the mathy part.


Symbolic knowledge distillation constructs a high quality knowledge graph at scale.


Okay, that's their data generation process.


A critical teacher results in a higher quality student.


Now, granted, the critical teacher makes the quality of the data set better, and therefore


any model, the student that is trained on that data set, it will become better.