and the time interval over which you were choosing to drive or not is probably something on the order of hours and so essentially what we do is we say well kind of from a driver's perspective they're not responding to the instantaneous state of the system instead what they're thinking is if I enter what's the expected earnings that that I'll receive if I'm if I'm part of the platform and what they do is they compare the expected earnings they'll make while they're in the system to essentially be expected you know the


kind of a reservation earnings that they want to reservation earnings rate okay this is also really interesting so this is kind of a target earning model of the driver like basically they have some fixed mental model of how much they want to be able to make and they enter if it exceeds that you know I think Chris pointed to examples of a lot of different driver behavior in their data and you know you would certainly see that across all the platforms so I think that's that's a that's a really interesting direction also first of all

Dynamic Pricing in Ride-Sharing Platforms


just do you have the right kind of utility model for drivers I think this is when I say you know when I say that we're taking an extreme point what I mean is this time scale separation between drivers and passengers is a fundamental part of the model it would be interesting to think about what starts happening if drivers are responding more directly to instantaneous state and in particular I think the the way this comes together with the network's comment earlier is if I'm able to provide signals to drivers that say this is a place where I think


demand is locally higher than supply and what I'm doing is I want to induce drivers to move in that direction and when I'm trading off the effects I'm trading off are that it takes drivers time to move to a new area it takes drivers away from the area that they were in you know so I sort of I think that that's that's really where the the network modeling gets gets especially interesting so that's kind of one of the things that we want to keep doing with this work as with the passengers I'm going to model this reservation earnings


rate is iid across the drivers so similarly you can work out what is the actual rate at which drivers enter there's some exogenous rate drivers will enter if their earnings you know desired kind of reservation earnings rate is is lower than what they think they're going to make in you know per unit time and so expected earnings divided by expected time okay you don't have to worry too much about the fact that the expected czar both the numerator and denominator here there's sort of a waltz identity argument that lets you do away with that


passenger model yeah oh yeah sorry that's a great point so so you're right this is not the number of rides that actually gets served this is a number of ride requests and so what will happen is that if there's no driver there your ride request will be blocked and it'll be dropped no but what I mean is might decision as to whether or not to even make the request that my available drag yeah yeah so so that's a really good point so first of all when I said earlier that I wasn't dealing with ETA that's what I meant that I'm not


modeling so in queueing models where you think about and you know this is not necessarily something specific to me I'd say that this type of trade-off exists everywhere let's just think about like so I'm using a model where essentially the passengers cost is blocking right they may not get served well in a real queuing system like blocking and delay are not completely dissimilar from each other in the sense that if I get blocked but that really is saying is I just have to wait longer to get done with whatever I want


to get done right and so I I think like the right there's two ways you can think about blocking it's like actually what the cost is it's I tried to get a ride and I couldn't say New Year's Eve or you could think about it as I didn't get the right and that means I have to wait you know longer to be able to get a ride and and I think I think if this is a very very course sort of 0th order approximation to the right thing to do and the right thing to do would be to actually include ETS in the model and model that yeah that's a good


question okay so that's the drivers so let me just quickly run through what the queueing model is basically the queueing model is now when I be my queueing model isn't in describing this to you I'm not going to say anything about why drivers and passengers are coming in at the rates that they're coming in they just are and then let's so I've already told you that I have some model that's strategic for how I'll determine the rate at which rides are requested how I'll determine the rate at which drivers actually entered the system now let me


tell you sort of what actually happens inside the model but those you know rates of arrival of drivers and passengers so basically drivers enter at some rate lambda and when there's a drivers available ride requests arrive at some rate which depends on the number of available drivers that's what we computed earlier um if a driver is available the ride is served otherwise it's blocked rides lasts an exponential time with mean tau in the network model this sort of involves a random walk around the network so it's a little more


complicated than that and after right completions this is another point where I think someone could do something interesting with this so after ride completion there's some exogenous probability that the driver signs out or it becomes available okay so another thing that could be interesting that you can do here is make the exit probability dependent on their experience in the system for obvious reasons that leads to a much more complicated game so this is a you know far simplified version of that I would say that I'm actually less


sort of concerned about this assumption that this is exogenous because drivers are still making entry decisions based on their kind of expected earnings in the platform so it's not as if they're completely ignoring how they're going to do but I think if you wanted to refine it a bit you could think about making this something which is endogenous okay so that's that's basically the the model of what's going on and so if you look at the picture of what's happening here there's two reasons that there's available drivers there's either new


entry from the outside world or there's a driver who was busy who became available and and came back into the system who chose not to exit okay so there's some queue that's going up and down available drivers coming in and then when rides are requested this cue gets served down okay so a major sort of simplification for us and one of the reasons for exactly the lining of the assumptions we had is that this type of queueing model turns out to be what's called a Jackson Network and this is more generally true for the


network model and Jackson networks are great because they're steady-state distribution is a product alright so despite the fact that there's all these dependencies in the queuing network the the steady-state distribution has the property that in steady state the queue lengths in the different you know parts of the network look like they're independent okay so in particular here what it means is that the number of available drivers is something which we have like an exact expression for the for the for the steady-state distribution okay so I'll


skip this slide I sorry I won't go through everything in detail what I mean is I just want to tell you what an equilibrium of our system is so an equilibrium of our system involves basically saying I connect together the strategic behavior of the passengers and drivers the pricing policy and the queueing model and so what I do is I say okay let me take the queueing model compute a steady-state distribution from it using the steady-state distribution for every driver I can work out what is their expected earnings that they'll


make while they're in the system and what's the expected time they're gonna live I can use that to then work out what's the entry rate of drivers and then for every passenger I can work out you know sort of for the passenger kind of the steady-state condition is easy it's the same one we had before that passengers will enter as long as the price is going to be is going to be lower than their reservation value and that goes in the entry rate of ride requests so all this comes back together again and I need a consistency check


that the steady-state distribution actually came from these things that I just computed alright so you can do that and that's kind of a definition of a system equilibrium in our model and you know what we show is that basically as long as you have very mild regularity conditions on the system namely among other things that the price increases when the number of available drivers decreases so this is a condition on the pricing function then equilibria always exist and their unique under reasonable sort of smoothness conditions so oh we


go that that is so let me talk about sort of nicely smooth those conditions here I mean on the distributions so let me let me sort of move on to how we want to use this so I guess you know part half the talk is like here's a model that sort of sits on some knife at knife edge of tractability and still capturing a bunch of effects that are important I've already pointed out to you at least five or six different ways that I think you would want to do things to the model to capture effects that are that are important in practice that said at least


in our experience thinking about it adding any one of those things made the model much harder to work with so in order to in order to make progress beyond what we had I think part of the question is going to be you know where you get technical simplicity despite having at it in these communities additional complications and I think one thing for us even with the model that we had that is very helpful is to use awesome products to simplify the analysis this is something which is now by now very standard in in analyzing


these types of matching markets Eric bhootish and Eduardo Acevedo have a really nice paper that sort of talks about some of the things that you can do using this type of approach so in our case the the limiting approach that we use is that we have a sequence of systems we consider in the end system I'm basically scaling up the exogenous rates the arrival rates of passengers and of drivers and in the end system I have some pricing policy that's actually indexed by in okay and I'll talk to you a little bit about how that how that


works so in each system this gives rise to a system equilibrium and we basically analyze pricing by looking at the asymptotics of these equilibrium ok so that's what I'm going to show you I'll show you a bunch of pictures that explain what's going on and there's corresponding theorems behind it so let me start with static pricing again static pricing doesn't mean there's a single multiplier the whole day it means that I have a predictable uncertainty on the order of hours or something like that that I use to set a multiplier but


I don't change the multiplier on the basis of the exact number of available drivers that are in the system so but you should be thinking when I say static is that idiosyncratic stochastic fluctuations are not being captured by the by the dynamic pricing policy so so in math this means that P of a is a constant for all a that no matter what the level of the queue is I'm setting the same multiplier so there's a theorem here that you know don't worry about reading the technical sort of expressions carefully but here's


basically what's going on what it says is that you pick the multiplier that you're going to use at all eh okay once you've done that then if I scale and there's there's this should be scaled by and this should be our n over and its a scaled completed it's obviously the rate of completed rides is going to go to infinity if if n goes to infinity it's the scaled rate of completed rides that that's normalized so if I take our n over n the rate of completed rides divided by n that has a really natural sort of interpretation


imagine sort of two unconstrained systems one in which as a passenger whenever I request a ride there's always a driver waiting for me all right and another in which as a driver whenever I finish driving there's always a passenger right there requesting a ride that I can get matched to okay so in one of them demand is not a canoe in the first system I just described supply is not a constraint the second one demand is not a constraint so each of these two sort of naturally give you a supply curve in a demand curve this expression


is basically how many drivers would enter if they knew that whenever they were available there would be a passenger immediately there for them and this expression is how many passengers would enter if they knew that whenever they requested a ride there would be immediately be a driver there to serve them that's the same entry rate we had before and so that's that's what put is basically available the throughput is a min of available supply and available demand so that's really nice because you can visualize it as sort of a demand and


supply curve crossing each other this green curve here is the demand curve this green curve here is the supply curve they cross right at this point now what's the x-axis this is different than a usual plot in in economics the x-axis here is price the y-axis is throughput so I'll tell you about the red curves in a second so what I'm saying here is that you pick a multiplier on the x-axis and the the actual throughput that you're going to see in the system is the minimum of this green curve and this green curve okay so this sort of pyramid


shape thing is what the overall throughput is as you vary the multiplier and you can see that if your only choice was what multiplier to pick and you didn't have any dynamic pricing available the multiplier you would pick is the one where they intersect right that's the peak and so that's that's what we're gonna call that the balance price for the rest of the talk the red curves are just depicting what happens as n increases all right so these are simulations for finite n and the green is the theory in the in the limiting


system all right everybody okay with that so basically what I'm saying is that there's like a natural notion of available supply and available demand and I can use that to say that if what I want to do is maximize throughput then the multiplier would pick is the one balances the two of them it's pretty intuitive it's just nice to get that out of the out of out of the primitives so I'll skip this slide I just said that so now let's talk about dynamic pricing and what we did is we decided to focus on a particular family of dynamic pricing


policies there's there's a small part of the paper that says something on slightly more complex policies but it's actually it is actually a little bit harder to deal with sort of arbitrary dynamic pricing policies and come up with exactly the same conclusions so what we do is we focus on threshold policies and and these are very natural because they match exactly with what's done in practice which is basically that I have a threshold and then if the number of available drivers drops below that threshold I set a multiplier that's a high


multiplier if the number of driver available drivers goes above that threshold I set a multiplier that's a low multiplier okay so I'm kind of moving between these two multipliers the slightly more general thing we looked at is a finite number of thresholds okay and then you know the more general thing obviously would be that the the price curve can be anything across the number of Avila's and so this has kind of inspired the reason we specifically focused on this is because we wanted to try to say something about what search


and prime-time policies are actually doing and they basically have this flavor that there's kind of thresholds that when they're crossed the the multiplier is going to go up or down as a result okay so I think I can convey kind of what our theorem is with pictures more easily than the results I'll show you in a second so let's let me just sort of parse this picture for you okay so on the x-axis sorry on the x-axis what I'm plotting again is the multiplier right so it's going from from 0 up to 5 here that's the price


multiplier that I'm going to use the blue curve is the throughput that I obtain if I just set that multiplier regardless of the number of available drivers same basically as the curves that were showing you in the previous graph which is pure static pricing okay the purple curve is what I get with a very particular dynamic pricing policy the dynamic pricing policy is one where one of the two multipliers is set at green at this at this point right here and the other multiplier is the one that's bearing on the x-axis okay now


what does that mean if I'm on this side then it means is that the high multiplier is this value and the low multiplier is down here and if I'm on this side it means that the low multiplier is this value and high multiplier is the one on this axis that make sense okay so that's that's kind of how you can do a 1d visualization of a family of dynamic pricing policies in this graph and that's also why there's a kink right here okay so this then the purple curve is the throughput that I get with that particular dynamic pricing policy as I


vary on the x-axis and what I'll do is I'll just make n larger and as n gets larger what you notice is what you should see is that the static curve and the dynamic curve the peaks are going to come together okay and so that's what happens in the limit here so this this is the limiting value with n going to infinity the purple curve is what I get from dynamic pricing again defined the way that I talked about it where this is one of the two prices and the other price is varying on the x-axis and the blue curve is the one I just showed you


a few minutes ago which is what I get from static pricing and what's the important point to notice from this graph is that both of them coincide at their peak okay so what that's basically saying is that in the fluid limit and this in this kind of hydrodynamic limit where we're scaling you know down by n that you're not getting anything by using a dynamic pricing policy over a static pricing policy you know there's a bunch of suppressed in these graphs okay number one it's numeric so obviously I haven't convinced you about other


potential values for prices that I could use number two I didn't say anything about the threshold right so like what's going on with the threshold and so in I'll show you the theorem in a second one of one of the things we're doing in the background here is that as n is getting larger we're varying the threshold that we use and we're doing so in an optimal way given the two prices that are chosen right so we're kind of favorably biasing the dynamic pricing policy to pick the best threshold that we could given the two prices that were


used and so really the result is saying here is feel free to pick the threshold and the two prices however you want and you're never gonna do better than what what static pricing gets you yeah okay now B I should be done before that already okay so this is the limit and what it basically says is let our n star be the rate of completed rides in the n system using the optimal static price and let our n double star be the rate of completed rise in the n system using the optimal threshold pricing strategy then if the valuation distribution has a


monotone hazard rate I'll talk more about this in a second then the the difference between these two scaled by n goes to zero okay so some comments on this first of all I want to point out so there's this is a really important restriction in the result and the proof of the result what ends up happening is that we need to look at sort of how the level of demand changes as you vary the pricing policy and it turns out that some condition on the valuation distribution is is sort of necessary it's a little bit looser than this it


doesn't quite have to be that strong that that's probably most interpretable definition what's interesting is that there's no condition on the on the Preferences of the drivers FC does not enter into this theorem so FC could be arbitrary it's the the restriction is only on that fee okay that's one point the second point is I guess what I find interesting about this result it the way I would state it I I think like the the more glib statement is there's no value to dynamic pricing I think the way I would stated that's a bit more precise


is that there's only second-order benefit to dynamic pricing right so to the extent that you see benefits from dynamic pricing it's happening because it's happening because you're actually able to sort of correct things in the in the sort of a Gaussian term not in the not in the fluid term the third comment I want to make is that maybe one naive view you might have on this result as well of course because you took the limit you got rid of all stochasticity but the thing is that's not true so that the drivers went there is actually a


well-defined steady state distribution that's seen sorry/not drivers by passengers when they enter even in the limiting system okay so when a passenger arrives it is the case that the the number of available drivers that they see is a random variable and that remains true even when you pass to the limit okay so it's it is true that we're speeding up the system but from a passengers perspective they're always reacting an instantaneous state so that stochasticity is still relevant and then i think that's part of what makes the


makes the fact that this happens a little bit surprising so one comment i want to make is that these types of results are similar to stuff that's kind of long-standing in the revenue management literature where in the early results in that literature you know it was established that that fluid pricing policies of the sort that we were talking about are not going to be better than dynamic pricing and there's been a lot of work extending those types of results but one interesting thing in that literature is they're not really been any work that


looks at two-sided markets so that literature usually what happens is you're kind of given a fixed basket of stuff to sell and then you're allowed to use whatever mechanism you want to sell up to a deadline and like the canonical example is airline tickets so you're an airline you're selling seats and so you have like planes that are empty and you have a bunch of seats to sell up to a deadline when the plane actually takes off you're allowed to use whatever pricing policy you want to set prices for the seats okay in our case kind of


one of the things I think that makes this really interesting is that we have this you know that the seats are strategic the seats are the drivers and and they're they're entering because of because of what we're doing so you know that's one of the big differences and so again just to point out kind of why this is interesting to work on is I think there are if you look at the problems that are faced by many platforms in managing inventory internally so the analog in air B&B would be sort of managing the inventory of hotel rooms


they look like classic revenue management problems but with strategic supply and so that that you know that perspective is something which opens up I think a range of algorithmic insight that isn't there yet in the literature so I'm going to skip the proof just because we're already well over time for this session I'm happy to talk to you about that offline if you want or if you play back the video and look at the slide slowly you should be able to make it out so last thing I'll tell you is and this result I think is less


surprising it's just I just like it because of the nature of how we formulated it so dynamic pricing is obviously helpful all right and and already there's a hint in this second-order effect that I mentioned and really like why do we use it well one reason we use it is because we don't want to have to sit and futz around with what we think the system parameters are actually going to be even when it's predictable uncertainty all right so one thing that's nice about dynamic pricing is it sort of naturally adjusts itself


to wherever the system is living okay and somehow you would like to be able to make a statement that that captures that robustness that if the system parameters are known yeah then maybe static pricing is at least as good as there never is you know doing as good as dynamic pricing but when system parameters are unknown presumably you should gain something in robustness because you were using a dynamic pricing policy so there are a lot of different ways to give these kinds of robustness results and again in the revenue management


literature it usually involves taking the second-order limit like looking at a fusion limit of the system and there those are nice but but definitely technically complex what we want to do is take maybe a more of a robust optimization viewpoint where we basically said let's ask can we get sort of dynamic pricing to be near optimal across a wide range of system parameters okay so again I'll just tell you that the result in pictures so here what I'm doing is I'm picking one of the parameters let's say the exoticness


arrival rate of passengers and of you know app opens and what I'm plotting here is this is the throughput I would get if regardless of what the you know for each value the system parameter I set the optimal static price okay this is the blue curve is what I get if I set the optimal strategy for static price believing that the exogenous rate of customer arrivals was mutant was a 4.0 but in fact it ended up being something different and so yeah right here obviously it's it's optimal but then it really degrades quickly as I move away


right well you really want like is the difference between these two things now suppose that what I tell myself is okay you know I don't know what me zero is going to be exactly but I think it's gonna be between three point six and four point four so let me take each of those to compute the corresponding static prices for each of those to the optimal static prices and use these two as the two prices in a dynamic pricing scheme okay again buried in the background is what threshold do I set to move between these


two and I'll just tell you that that we're using the same sort of optimality result that was in the proof to be able to set that threshold between once you give me the two prices I can set the optimal threshold right and what you find is that if you now plot how dynamic pricing does using these two prices the two prices from the extremes then yeah it degrades badly outside of that uncertainty set but inside the uncertainty set and what we can show is that dynamic pricing always gets you at least the linear interpolation between


these two optimal values okay and at least in the numerical experiments we've done that tends to be quite good this the the green curve is is concave and all the experiments we've looked at so far numerical experiments so basically the point is the result we have is something which characterizes the robustness of dynamic pricing through this notion of an uncertainty set over system parameters and says that you always do at least as well as the linear interpolation from the endpoints of your uncertainty set and this


kind of a technical statement of that okay so let me conclude so a bunch of different things that we were trying to do in this in this work and so some things I didn't you know managed to get to network modeling with multiple regions when I say our main insights generalize I mean they generalize sort of to the extent that we're you know able to characterize this difference between static and dynamic pricing we haven't accounted for ETA sort of in the way that David asked about right aggregate welfare is something like I


said numerically we get very similar insights but but we're trying to work the theory out there and I think it really gets interesting when you start changing you know when you start asking you know much more foundational design questions so one thing that that I think Chris even pointed out is that you know you can imagine showing drivers these heat maps where we're saying like here is a place where there's there's more or less demand available we have some model inside the queueing model we've we've come up with some nice sort of tractable


ways to deal with this but extending it then to add in the strategic the strategic component has not been so easy but I think that again this is a really important direction to take the work it's be structure changing the percentage we talked about already and finally I think that there's sort of changing the matching algorithm there's something which has really not discussed a lot if you ask you know uber or lyft I'm sure they would say that they want it to be fairly transparent so you're match to the nearest driver there's all


kinds of really good reasons why you might want to manage inventory very differently across the network especially if you for those of you that were here for the industry the reverse field trip day you know they were talking about this sort of you know tessellation of the earth into smaller regions than what they've been using before and once you start doing that I absolutely think you would want your match algorithm to occasionally pick people from adjacent regions you know depending on you know or another example


this would be the Chris on Monday mentioned that uber wants to be able to accommodate driver preferences it use either Chris or last week that if a driver says hey I want to be able to be get back to the East Bay in an hour I'm gonna accommodate well that's gonna change the match algorithm you're not going to give it to the nearest driver necessarily and I think this is again one of these things with sort of geographic matching you would have to have a good queueing model and you would have to be able to model the incentives


hi there what you're seeing here is an energy based model that learns the concept of a shape from a demonstration on the left so on the left you can see a demonstration of data point sampled from a shape in these cases circles or squares and then the corresponding energy function that the model in first from that and then it can replicate that shape on the right using that energy function so the paper we're going to analyze today is called concept learning with energy based models by yegor more Dutch of open AI and this is a very cool

Concept Learning with Energy-Based Models (Paper Explained)


paper or at least I think it's a very cool paper but it is also a very hard paper so therefore first I want to kind of make a bit of an introduction into the concepts that we are facing in this paper so the first thing you need to know are energy functions or energy based models what is an energy function an energy function sometimes called e is simply a function with one or multiple inputs let's call them X and you can make the if the energy function is happy with X it will be the value 0 and if the energy function is not happy with X it


will be a high value like larger than zero so this is happy this is not happy so let's give some examples of this we can formulate almost any machine learning problem in terms of an energy function let's say we have a classifier the classifier is takes as an input image here may be of a cat and the label so if the label is cat then the energy will be zero if the energy function is of course working correctly and if but if we give the energy function the same image but we give it a wrong label dog then it is very high in the case of the


classifier of course we can to simply take the loss function as the energy function and we are altom automatically get an energy based model so the loss function here would be something like the negative log probability of the of the sorry of the correct class but in any case it is just going to be a high number let's call it 10 to the 9 so the energy function says ha this is very this is very bad this thing here is very bad the entire thing you input it won't tell you yet what's bad about it so that also means you can


change any of the two things to make the classifier happy now usually we're concerned with changing the label it's like tell me which other label do I need to input to make you happy and if we make the labels differentiable of course we never input the true label we actually input like a distribution softmax distribution over labels and that's a differentiable we can use gradient descent to update the dog label we can use gradient descent to find the label that would make the energy function more happy so we could use


gradient descent to get the cat level if we had a good classifier but we can also we can also optimize the image to make it compatible with the dog label right that's things that if you ever saw deep dream or something like this those models do exactly that they optimize the input image for a particular label and there you can view the entire neural network including the loss function as the energy function so what's another example another example is let's say you have a k-means model and the energy function is simply input a data point


and for the data point what you're going to do is you're going to find the min cluster index the min Kay over you know you have your multiple clusters here and your data point might be here so you're going to fight the cluster that's closest and then the distance here this since Dee will be the energy of that so the model is very happy when your data point it comes from one of the clusters but your model is not happy when the data point is far away and that would be the cost function of the k-means function so that's an energy based model


too now currently energy based models have come into fashion through things like Gans or any conservative noise contrastive estimation so in a jet in a gam what you have is you have a discriminator and the discriminator will basically learn a function to differentiate data from non data so that by itself is an energy function so the discriminator will learn a function and that function will be low wherever the discriminator thinks there is a data right so it will usually do this around the data point so the data points form


the valleys right here and then the generator will basically take that discriminator function and will try to infer points that are also in these valleys to produce points that are also in the valleys and then you basically have an energy learning competition the discriminator now tries to push down on the energy where the true data is and push up on the energy where the generated data is and that will give you basically a steeper energy based function in the future I hope so in this case the discriminator neural network is


the energy function and the D generator just tries to produce data that is compatible with that energy function so I hope that concept of what an energy function is is a bit clearer and in any again any machine learning problem can be formulated in terms of an energy function now what is not done so far is what we alluded to a little bit before in the classifier example and also here so right now when we want to Train again we simply take the generator to whose data now what's the generator skull the generators goal is to hit


those valleys in the energy function and we produce a generator into in one shot produce this data but what we could also do is of course we could just start somewhere let's say here we pick a random data point and then we use gradient descent because the energy function in this case is smooth we use gradient descent to just drop down this valley and then find ourselves in this valley so without ever training a generator we can use this methods to produce points that are in the valley of the energy function right and this I don't know if


people I can guess people have trained gams like this the reason why it doesn't work let's say in the real world is because that procedure will just produce adversarial examples for the discriminator and those usually look like nothing like data because if you keep the discriminator just stable and gradient descent against it what you'll get isn't really qualitatively good but in principle if the discriminator was a good energy function for the date to describe the data we could use gradient descent the same up here in order to


find a good label for an image given that we have a good energy function right so this is that we could simply gradient descent on the label in order to find a better in order to find a better label so in this paper we're going to have a situation where we say we're given an energy function and we're given a bunch of inputs they are then called X a and W and if I have my energy function already if I have given my energy function and I have given two of those three things any two right I can infer the last thing simply by gradient


descent on my energy function because I know the energy function is zero when all of these when the energy function is happy with the input so when all of these things agree basically the energy function is happy it will output zero otherwise it will output a high value therefore if I given any of those two any two of those three things I can find a compatible third thing by descending and then of course over here in these machine learning problems the task was always actually to learn an energy function right so usually in the


training dates that we are given images and labels and we want to learn this energy function which would be parameterized so we want to learn the parameters and the same here in our general case if we are now given three things but we are not given the parameters of the energy function we don't know what those are as long as we're given all of the inputs and our training data set and our training data set guarantees these are actually you know these are inputs that are compatible with each other the energy function should below we can


simply gradient descent on the parameters of the energy function so in a sense there are four things right there are these three inputs and then there are the parameters of the energy function if we're given any three of those four we can gradient descent on the rest and that's going to be the basis so the X here is going to be the so-called state and the state in this paper is going to be images of entities the entities sorry is not going to be images but the entities are these little circles that you're going to see and


each of those entities can have an exposition a Y position and I believe a color so our G and B so each of those can have that and then the concatenation of all of those attributes is a one big vector and that is your X that's your state so state is number of entities and their attributes a is going to be an attention mask over the state so a is going to be here you have four entities so a will have four entries telling you which of these entities you should pay attention to right now and W is going to be a concept vector so called so W is going


to be the embedding of a concept now what a concept is in this case is very general I can give you an example one a concept is do any of do the entities that the a pays attention to are they close to each other so in this case you see we have two entities that a has a high value one and this is this all up here and this ball down here now if the concept vector is the embedding for the concept of being close to each other then the energy function would be very happy if those two things are close to each other and it would be very unhappy


if those two things aren't close to each other but in the very same situations of the same X the same attention mask but a different concept so a different W vector right here then the the energy function would be maybe very happy if the two things are far apart and maybe unhappy if the two things are close so the question is always how are the three things that you put into the energy function compatible with each other and given all but one of these things you can infer the other so let's say you have a perfect energy function for this


this all of these for this situation you're just given the energy function you can trust it and you are given let's make an example you are given the X so you're given the state I'm gonna draw the state down here right okay this is the state and you were given the W and the W is the embedding it's a vector but the embedding space but the embedding is for a line right so the geometric the geometric unit of a line now your task is to find a the attention mask that will make the energy function happy and as you can see right here


while you would do is you would put a lot of weight on this this this and this ball and no weight on that ball because those make a line and since everything here is differentiable so the state is differentiable the attentions differentiable and the concepts or vectors that are differentiable you can use gradient descent to find that another example if you're given again the same W so line and you were given this following thing and you were given now you're given the attention on these three and you say please find the X


please find the X the states that makes this energy function happy now this here you would call the starting state the x0 your ear task is going to be find the x1 find the state how do you have to change this state such that the energy function is happy and of course the answer is going to be is to push this ball here inward until it is in the middle of the two others so the three form a line right these three form a line you don't you don't have to do anything to this ball up here because there is no attention on it and the attention it's


only is the concept for the things that you put attention on yeah and the state are those three in agreement and the energy function is happy okay we have covered the basics now let's dive into the paper I think this is the longest introduction ever but I think it will pay off once see so they they specifically or this this author I think it's a single author identifies two different things that you can do with an energy function here of course you can do more as we saw but they identify two so here is where you


have given the initial state and an attention mask and you want to find the x1 the state that satisfies the concept and the tension the most this the author calls generation as you can see here these four things that you have the attention on are pushed around until they make a square because the concept right now is square and in the other case where you are given this x0 and x1 just call this X right here just call this thing X if you're given those two and you are given the concept Square and you're tasked with finding a the


attention mask of course you're going to put the attention on these right here and that is going to happen through gradient descent again we're not learning a model to give you that attention like in again we're learning a generator to just one shot give it to you right now what we're going to do is we're going to gradient descent optimize on our smooth energy function to give us that perfect attention mask that satisfies the energy function all right so this is the difference right here gradient descent is part of the output


procedure of the model usually we just use it to learn and we learn a one-shot model but here gradient descent is part of the model so they introduce energy functions here and they say okay we can have a policy on X so if we're given a concept W and if we're given an A we can have a policy over X which basically means we can find X's that are compatible with that by running gradient descent here you see there is an XK minus 1 and we are running gradient descent on the energy function with respect to X to find a better X


that satisfies the energy function given those inputs and the same if we want to find an attention mask we are running gradient descent on the attention mask again in order to satisfy the same energy function so you see the inputs are both times the same the concept here we can input square here we can input square but the difference is what we're running gradient descent on and what we keep constant and I would get I would add a third line here actually because we can also if we're given an X and an A we can also infer a W and that's going


to be an integral part so if I have this right here and this situation and I have say I have a tension on these for now I can ask the model so I'm given X and I'm given a I can ask the model to infer W and the model should ideally output ha this is square now the model isn't going to output square the model is going to output a vector representation of square right so the model is going to output square but as a vector of numbers because that's how we've trained it W isn't it is the embedding but what we can then do later is we can say okay I'm


not going to tell you it's a square you just come up with a vector W it describes this situation and now I'm going to take that vector W that you came up with miss mister or missus model and I'm going to take tell you a new situation this situation right here and I'm going to now give you X and I'm going to give you the W that you yourself have output and now please tell me what's the a and then the model is of course supposed to tell you oh these four here or the a so without without ever telling that it should be a square what


you can do is you can let the model infer a W from one example situation and then transfer that W to a new situation so it can identify you can just say whatever concept I have up here please apply that same concept which is the W down here and this is the entire paper now this is the concept learning through energy based models okay so that is kind of a third line I would add down here you can infer a concept vector if you're given the X and the a so in order to do all this their energy function is going to be a so called relational neural


network so what you'll have is you'll have a simple neural network a multi-layer perceptron that always connects two entities to each other with the concept vector and then this is a belief a sigmoid that connects the attention masks of the two and then you simply sum over all pairs of two entries in your model and then you send that through an MLP sorry through an MLP again this I believe is not so important it's just important that they can feed this entire situation the X the a and the W they can basically feed into a


neural network in the neural network comes up with a number of how well those three things fit together and then you can transfer these concepts that's pretty cool now the only question is of course and we've always said we're given an energy function or just we just have it but of course this is a neural network and the neural network has parameters and the parameters we don't know what good parameters are at the beginning so we need to train this thing and again the reason why these are toy problems right here is I mean we'll get


to why it's computational but this is kind of a new field I believe in machine learning at least I come from classical machine learning and we only ever have used like SGD to train and we only have her have produced models that one shot produce something and here we this is a I believe there's a new concept where you use gradient descent as part of the output and that makes a lot of trouble and so that's why we work in toy problems so what this this here is the situation I described you have a demo event where you're given the X and


the a and you're supposed to infer the W so the question here is what's the W and the model will come up with a W and you're not gonna do anything you're not right now you're simply gonna take that W and tell it oh well here is a so called test event so please apply the W you came up with in this test event and please find me the the a in this case that satisfies the W and the X I give you here and of course the a right here is as you can see even you don't know that it's a square and the actual concept here is move the grey ball to


the middle of the square right that that is it here but no one has told me this I just looked at the picture so the the correct answer here would be to place attention on those four things and then to take this thing and move it to the middle right here in in the in this over here so that would be the correct answer now the question is how do you train something like this and they they show that they so this is the loss function right here the loss function is they give you a concept and an initial situation and you're supposed to infer


the x1 and the a and the loss function is simply the negative log likelihood of that but what does that mean so will will make it easier if if you have this this procedure right here where you have demo event this up here this is demo and this is a test event how are you going this entire procedure how are you going to learn the energy function well in this case this entire procedure this entire thing is one training sample sample but usually we have input and label and now here it's much more complicated because so we have input


okay that's this X and this a cool but then we have SGD as integral part of the procedure to determine the W and now what we could do is just apply a loss to the W but we don't because we don't know what the embedding space for the concepts is we could maybe train a classifier but in this case we want to train the ability to transfer these concepts so our training sample needs to be one time transferring a concept so SGD for one is part of our process here and not only that but then this this X here of course is also part of our


training sample write this up here as X 0 and this here is X 1 and now we need to find this a this attention mask and that is an SGD again remember inferring anything through the energy function is a gradient descent process so ultimately our one training example consists of X 0 a at the beginning so let's call that a zero it consists of the SGD procedure to find W it consists of X 1 and they consist of the SGD procedure to find a the a 1 the output a and then that will give us the output a the a 1 so this here is our input in the classical


machine and this would be our X and this here would be our label Y and that's what we trained on we trained so it such that the output right here the this is of course sorry this is of course the Y hat this is what we predict and in the training sample we just write a little generator that will you know make this situation that knows what the concept is right it will say okay I'm gonna make an example for a square then it make this will make the attention mask for a square and then it will make the new situation again with a square


but not tell us the attention mask there and it will make the attention mask into the true Y so at the end we can compare what our model output the attention mask we output here without ever knowing that it should be a square and we have the true label which comes out of the generator that at the beginning decided that it should be a square and then the loss in the distance between those two that's our loss this is an in this is an enormous procedure to get a loss and most crucially you have to back propagate through optimization


procedures and this is something that we just can't do yet in our models if you take an image a resonate 50 right right now we do one forward propagation to get a label in this procedure if you had two back propagate through the optimization procedure for each sample you would need to basically back propagate through 50 forward passes of the resonate if you if your optimization procedure is 50 steps long and that is just not feasible right now so that's why we don't do it but I believe maybe once we find a smart way


of back propping through optimization procedures a whole lot of these things will become the new and new wave and machine learning I really I'm excited - I'm pretty sure it doesn't work yet and this is very figley fiddly work but I'm excited by the prospect that we can do this so this is the training procedure right you've given X 0 x1 and a and you Optima is in order to infer the concept behind it right the generator that your level generator of your training data it knows the concept it has a concept in mind when it


generated this but you're not telling your model what the concept is it needs to infer that and then using the model thing that the model inferred you can either give it x0 and x1 and infer a or you can give it the X and the a and in forex you can do either of those right these are called identification or generation respectively and then you compare the output here to what the generator at the beginning thought again it's not telling you it's that's because that's the label and you compare this to that and that will be your loss to train


your energy function parameters so your training samples if you think of this entire thing as one forward pass of the model then it's just classic machine learning right you have a training sample which is one forward pass and you have a corresponding label that you infirm so let's jump to the experiments right here experiments are actually pretty cool so what they've done is for example have taken the concept of being far apart from something now being far apart so that the little X needs to be as far away as possible from the ball


that has the attention on it so if you do generation and you start the little X right here and you ask the model where please infer the next state of the world it will push that little X away right here and in color you can see the energy function valleys of the position of the X so it pushes it away from this thing but if you take the same concept embedding the concept embedding of being far away but you don't do generation you do identification which means you infer the a then it will simply tell you that this ball right here is the furthest


away from the X right so you can do all sorts of things like this and transferring concepts I find this here pretty interesting so they had to have two different concepts one concept is read as an identification you need to identify the red ball but the other concept is you need to turn something red right you need to take a ball that is maybe now blue and of course the color you can gradient descent on the colors you'd need to make it red and since the energy function it just takes three input X a and W it doesn't you you you're not going to tell


it right now in which situation you are it has to create create this W embedding space through learning and if you do it with those two concepts then it will put the make something red concept and the is something red concepts in the same places so this is a PCA and then blue I think these blue is the attention codes for identify the red things and in red or the generation code for make something red and they will be put in the same place which is pretty cool it means that the energy function really learns the feature of something being


red I find this pretty pretty neat and then here they they have some experiments where they basically show we need that gradient descent optimization procedure because only after many steps will will the energy function basically be aligned with the concept that you want so if you have a zero shot model like just one forward pass as we do here you'll see that the energy function that is supposed to make a circle from samples right this is the example concept right here it if you just have a one shot model it will it cannot but in this case


at least it doesn't learn to one shot produce only if you opt in for a few steps will it get this so you optimize at inference time and that seems to be very important you can see again here demonstrations of this so the example is this and then the model as you can see after 20 steps learn optimizes the points to go to these locations whereas after only one step it didn't do that yet so there are complex things at work here and this column here is where you don't have a relational or neural network so you can't basically


capture dependencies between things so you you have no chance of making a square because you don't know where the things are in relation to each other but that's more of an engineering question their point is basically that if you have models that do an optimization at inference time they are much more powerful than models that just do a one-shot forward pass it's sort of like an auto regressive model in NLP versus a non auto regressive model that produces all words at once if you produce all words of a sentence at once no word can


depend on any other word and you can just come loose independent or you can just produce independent things which will make the sentence often not make any sense they also have this KL objective which is a regularizer which I believe that's just a trial and error they built it in because but it is a regularizer I don't want to really go into that and then they they do demonstration in and they re-enacted on a robot the demonstration here is that there is a situation where two things have a tension on and you're supposed to move


something into the middle of the two things so that's the content you don't tell the robot the concept it needs to learn that from data and then infer that this is the concept that you want and then transfer that to the other environment now you know this you look you know there's this robot environment but ultimately they still encode the positions of these things and the position of that and really all you have to do different here is that instead of moving this actuator directly you need to like calculate what you need to do to the


individual joints in the robot so I think this is maybe because it's open AI and it needs to you know look robot II and stuff but the problem here is not really different it's it's not even it's not real-world transfer or anything so yeah let's let go through some of the things they can learn with this so you can see here they can learn these regional geometric shapes and so on the left is the example event that the model needs to take the concept from now this is this is I believe very much identification so what


they did is they trained with a data set where all of these appear right so this there are squares there are lines there are circles so this is maybe my criticism here that it is not so much to generally infer a concept it is more like identify the concept so the model basically just needs to decide is this line is the circle or is this square because that was those things were in the training data set it would be nice to see how this generalizes to general concepts or if we can even make that if we can have a zero shot concept


inference and then transfer those concepts to other things maybe that's already happening I don't I don't know so here the spatial arrangement is to either be close to something or to be between two things so if the attention is on two things you want in between so you see the top ones are the demonstrations it needs to recognize the concept and it needs to basically optimize to fulfill that concept shapes so to make shapes is mmm oh yeah there's a triangle right again this this this just very much I believe


relies on recognition and not actual understanding of what a triangle here you have proximity being closer being far apart what else is cool oh yeah if the recognition for the same task right you need to identify the ball that is closer for and here you really also see the optimization procedure in action where for example at the beginning of each flicker you kind of see the attention being everywhere and then stabilizing to one or two points so if two points are equally close or far apart you'll see the attention being on


multiple points which is pretty cool right so that means the model really learns this this is concept here's the count quantity so you can either have one two or larger than three or something yeah that seems like they tried three and four and didn't work so they just said I will just do larger than three and here is this robot thing where it also always needs to move in between now this this is the part that I'm not really impressed with but you know whatever whatever you want okay I hope this was a good introduction to


hi there people so a lot of you have asked me how I read papers and honestly I don't think there is any super special method to it but you know I thought because people have asked me to make a video on it so I'll make a video on it and I'll try to share my method of reading papers and hopefully this is going to be somewhat of a miniseries or a series where I every now and then discuss how I read one of the papers that I make videos about and I'll try to select them such that different things are highlighted now I've selected this

How I Read a Paper: Facebook's DETR (Video Tutorial)


one right here for really for no particular reason other than I sort of remembered it and going to try to go with you through how I read this and how I encountered this and kind of try to honestly share what I thought at the first time when I read it and I hope this helps some of you if it does help you and if you like content like this of course feel free to share this out and subscribe if you haven't seen my original video on this paper it might be worth to go watch it I'll link it and with that let's dive in so again this


might be not really something new but I'll just go through it okay so first thing I do is of course read the title so the title has three parts and to end object detection with transformers so what I notice that I do myself is I like through reading a paper it's like read the paper with an open mind I don't do that I almost immediately form an opinion and a hypothesis of what's going on like so I see transformers so I know what transformers are if you don't I've made a video if made lots of videos on transformers attention is all you need


is the base paper for that so I know what a transformer is okay and I know that transformers are usually in NLP are usually used in NLP door there are things like you know other things with transformers but usually an NLP model then I read object detection and I know object detection is a computer vision tasks so immediately this here is sort of a a difference and I immediately try to assess what's the new idea in this paper and in this case it might be okay it might be applying transformers to object detection but


then I also see end to end and the only reason to put that in a title is because that's the novelty because usually in deep learning we're sort of used to systems being and to end and even if they aren't if most systems aren't end-to-end a lot of people don't is like end to end image classification on image net like Thanks so I I was guessing that the reason they put in end-to-end into the title was because that's actually something that's special about the model so now I have like two competing hypotheses of why this paper matters


first of all because it does it with transformers and second because it does it end to end and the of course the true fear is that the combination of end to end transformers all of that is what makes this model and I already form like a hypothesis of whether I like this or not like I have to be have to be honest I have very quick judgment of papers of whether I like them or not and then I sort of catch myself each time and I still try to so they're for most papers actually that I have sort of a negative opinion at the beginning where i will-


there are papers where I think like there is no way this is going to you know work or something like this I'm actually positively convinced throughout the paper so for most for most papers where I that I read I'm trying to find the positive things in there but I do form an opinion pretty quickly usually alright so the second thing this part right here I like I don't even see this is like advertisements on on like Twitter I like you you just I have always had issues with author names like people would come to me and be like oh


have you seen the new vineos paper and no clue and then when they say like oh that's where they use this character level model to do that and I'm like oh that paper so I like do not care who the authors are of a paper like I don't I can't remember the papers by their author names I've gotten better at it I have to say but i-i've always had trouble with this now that's not to say that a name doesn't pop out to me like if this would be like it like yoshua bengio or some someone like really famous then of course that would catch


my eye and but I also know that you know yoshua bengio is paper yoshua bengio slab is huge so just because a big name is on the paper doesn't mean that the paper is going to be of any good or bad quality sometimes the author's give you an indication of what kind of research is there like if you see a Najaf Clun or Ken Kenneth Oh Stanley you know that there's there's going to be the certain type of of you know learning to explore and and and kind of a bit more out-of-the-box thinking in their papers which I really


like but it doesn't immediately give you a clue maybe if you go by first authors it's much more indicative if you have already read some of their papers but most often I just ignore authors and go on the affiliation sometimes matters in in that it's a bit of a vicious cycle if there's a big name affiliation like Facebook AI Google ai and so on these papers also they get more exposure in like the press and so on so whatever Google a publish that paper all of these all these pops eye magazines like verge and this and life hacker and hacker news


and whatnot they like like write a blurb about it so often they get much more scrutinized for these papers they get much more they get much more the public attention but they also get much more scrutiny which in turn means that there is a bit more pressure on them to do good experiments so that biases meet like a little bit into the direction of believing their experimental evidence more now usually this is also backed up by the fact that I am actually convinced by their experiments usually so these these big-name papers often I find


myself even without or disregarding the affiliation to be convinced more than of like regular papers my most often issue with papers is that I don't believe the experiments and I make no difference like even if it's Facebook I still my prior is the experiments or crap and I don't believe them and they have to convince me of the opposite but some like I can't say that it doesn't affect me that it's like a big name affiliation okay so then the second thing is I sometimes I see the paper on archive and I skim the abstract


sometimes the abstract is informative and sometimes not so here it's like blah blah blah a new method that views object detection as a direct set prediction problem I'm like oh yeah okay so streamlines the detection effectively removing the need for many hand design components like non maximum suppression yada yada yada the main ingredients called detection transformer asset based global loss that forces unique prediction via bipartite matching and the transformer encode or decode or architecture so they make it clear here


why it matters and that's that's what I what I want to get at is sort of what's the new thing in this paper most papers are even though they're all very long and have lots of math and so on they often have like one or maybe two new core things that they really tell you sometimes zero but a lot of times it's like one thing that they really do and you you sort of have to but they're trying to cloak it often because they need to make their research as impactful as possible right but you need to sort of figure out what it is they're doing


they make it fairly easy for us in that they say okay they remove the need for many hand designed components like non maximum suppression which tells me that they're building something that's easier than what came before them and that already tells me it it's not necessarily going to be better their argument is more that it's going to be easier right there there are sort of two kinds of experimental results the ones where you try to beat what came before you and the ones where you're trying to say look our thing works just as well as this other


thing while being more advantageous in some other metric so I would place this already in the sort of second category and then they say what are the actual ingredients it's a set based global loss that forces unique predictions via bipartite matching now I at this point I know what these terms mean but at this point actually don't have to know what the terms mean what I need to recognize is that I simply have to go later and figure out what that is and a transformer based encoder/decoder architecture okay so there are two


things right here that I remember I need to pay attention to later there is this loss which seems to be special and there is the transformer architecture which seemed which they say okay that that's the model basically consists of those two things and then they have a short description of what it does given a fixed small set of learned object queries that your reasons about the relations of the objects and the global image context to directly output the final set of predicted in parallel that almost tells me nothing of yeah okay the


model reasons maybe this in parallel is something but the model is conceptually simple and does not require specialized library unlike many other modern detectors this sort of repeats this enforces my hypothesis that they're going with the hey this is a much easier way of doing things approach detter demonstrates accuracy and runtime performance on par with well established that further confirms my hypothesis that this is on are right they runtime performance on par with the current state of the art and at the end they say moreover data


can easily be generalized to proceed to produce Panoptix segmentation in a unified manner we show that it significantly outperforms competitive baselines training code and praetor models are available as part when I first read it is like ok can easily be generalized to produce this Panoptix segmentation this is I didn't know yet whether this is like a central claim of their paper that it can do this segmentation or whether this is like an added benefit to their paper because you can read it in both ways and I'm just


ready to find this out in the paper now after I've read the abstract and sort of already form the hypothesis of what's going on so here I already in my mind I already sort of have a model of how would I do that right how would I how would I do that and then what would I do so right now what I might be thinking is if I have a transformer over images that directly outputs the the predictions in parallel I'm imagining like an image and the image somehow needs to go into a transformer so maybe there's like an encoder like a CNN encoder that gives me


image features and then it's it's so maybe you sample this down this image this is just me hypothesizing what could be going on right and then I might be unrolling that right this image into a vector of these lower pixels and then so in my mind what I would do right here if without knowing anything more would be to do something like Bert's pan predictions so I would have Bert right here and I so for I would input this sequence right here and then to detect an object I would sort of think that maybe the Bert you know Bert has an


output that is the same length as the input right so it's it's very good at sequence tagging and things like this so maybe how it detects an object is going to be that it sort of like tags the tags the center location in the pixel of an object right here or a tag somehow the corners of the of the bounding box but then I don't know how this is going to be in parallel maybe Bert outputs like a score for each location and then you do some kind of matching right here so this is my initial hypothesis of what's going


on and then I scroll through and honestly the first thing I do is I go and find the pictures and know no different in all like since since you first book you read that's what you do I go and find the pictures because usually if someone proposes anything new that they're gonna try to make a picture of it luckily I don't do like super theoretical what not your Bayesian generalization bounds and I don't know so most often papers I read have some sort of picture and that's very helpful to me I know I know but yeah so I find


this picture and here I see okay you have image you have CNN okay gives you a set of image features or so far so good then transform or encoder decoder then set of box predictions so all of them come out here and I already read they're in parallel and then bipartite matching laws so here they I can see they color these in different ways and these color appear to match with these colors right here right in the green here and these they they also this is a very good graphic but from this I can already read that these here go to the no object a


lot of times the graphics aren't very good so this this is where I'm not saying in every paper you can learn by looking at the graphics like sometimes the graphics are terrible and you're like what's going on here I like I don't this this makes no sense this happens a lot in this paper right here this happens to be very very good explanatory graphics so I'll take advantage of that and I do the same thing in the other papers right but then later when it doesn't match what I read in the text I'll have to you know update my belief


and so on but here I see that these go to no object and this goes to no object so I don't know yet that this is the test set at the point where I read this I was sort of confused by this but I recognized that each of these boxes right here is going to be either resulting in a bounding box or in the no object prediction so from that I could conclude that these things here are maybe some sort of a fixed set right but I still thought that you know these that this would actually be the output of these image features so that in this


case you'd have like six set of image features and then you'd have like Bert here even though that's not an encoder decoder I still this was still my running hypothesis that somehow you'd map these image features to these boxes right here so and I didn't know what to what to make of this this thing right here so then I went through some more and look for more pictures and there are not sometimes I also kind of glanced at the formulas but okay when I Everest I see this this is just I mean this is kind of useless like okay cool you


minimize the loss thanks this okay didn't really pay attention to that ah new picture cool so this picture is much more informative than the other picture yeah I believe with the other picture they were trying to show case this loss how they do the matching and even though I could read a lot from that picture I did not get that part and that therefore I felt when I saw this and I just glanced at it I'm like wait what what's different then up here it seems like the same but okay let's look at this so again we see okay you have set


of image features that comes out of the CNN so that conforms with my belief but then this here goes into a transformer encoder and this comes out so immediately I see oh this is not the same as these boxes here right that was my hypothesis that these things here would be the colored boxes so I I say okay obviously that's not what happens this thing here seems to be sort of the encoded image information then that's somehow fed into here and that then there are these object query things and they seem to correspond to this so I'm a


bit more confused right now what I can see is that these then will result in these in these boxes okay so being confused by that I look for more pictures so I go look for more pictures and this here seems to be like of a visualization a lot of these papers have some sort of ablation experiments or so and so on this I just find a really cool picture for now I don't know yet what it means this I don't know yet what it means and I go down skip off this and then back here in the appendix I find this here which I immediately map to the


previous where this is the anchor and this is a decoder and I've already read the attention is all you need paper and at that point it clicked and means like this is not a Burt transformer this is one of these transformers that has an encoder and the decoder even though they told me like 50 billion times already I was too stupid until this point so now I know okay okay I see what's going on so the image goes through here and then this goes as a side input like as an attention from the decoder to the encoder like I know in NLP right so in


NLP this here would be a source sequence like maybe if you do translation and this here would be a target sequence so now whenever I see a transformer like this and it outputs something this I I look at it as okay this here is sort of the input that goes as like a side input over here and usually here you have the target sequence but that's not the case right here right you have these object queries so this is how far I get from the pictures now I go up so I have a sort of I have questions now I have questions and that's when I start


reading the paper only now do I start reading the paper after I've looked through all the images form the hypothesis and sort of have questions on how this works and we'll go a bit faster from now on - just not bore you with all the things so the introduction is often very important even though it's called introduction and maybe you know if you read a book like if there's like introduction or a prologue or something like this it's often kinda pointless introduction in these research papers is one of the most important points because


all of these papers they try basically all of them try to convince a reviewer to accept them and in order to do that they will set up their main points and their main story immediately in the introduction so what you'll usually have is a problem statement which is here like why what's what's wrong right now and then you have like a story of how their paper addresses the issue okay and that's that's here we streamline the training pipeline by viewing object prediction yada yada yada this is often formulates in words what


the paper is about and what contribution the paper makes right this is like a this is like a longer abstract the abstract is often very very cryptic very dense this here is often much more informative of what the paper does so for understanding the paper and a high level the introduction is the best place but given that I've already looked at the images and so on I don't actually draw many new much new information from this thing then is related work and honestly I I skip it like unless I'm the actual reviewer of a paper like when I'm


the reviewer of a paper I read the related work but often related work is just like you first of all you site a bunch of your friends and then you cite the mandatory papers and then you cite every single person that you think could be a reviewer because or you've actually been rejected from a conference with a reviewer claiming that your you haven't compared or you haven't cited data or that paper you can pretty much be sure that that's the if if it's not a glaring of may omission if it's like a niche paper and you haven't cited it then


you're like okay I'm gonna cite it just because the next conference you could be Mary viewer again so I'm not I'm not sure that these related work sections they're necessary like if someone wants to write their theses and they go and read this paper and they want references oftentimes this is a good place but a lot of it is just blah blah blah blah blah okay I know disagree with me if you want oh yeah - maybe - reading quality so I tend to at this point I tend to not skim so at first I skim but at this point I tend


to read every sentence and read it closely and understand it and when I realize like I'm tired or something I don't just skim the paper I've tried to skim papers and it doesn't doesn't work try to read every sentence understand every sentence and okay if you don't understand it don't stop reading because of that but try to not skim and be like oh yeah yeah yeah okay I gotta go to go to get away that is not helpful except related work skip completely cool then a lot of times in this paper now is the the model and this is the section I'm


actually interested in right so I read very very closely here and then I find out what their their loss is all about and again I stress read these things and understand them right sometimes it's hard but if you're if you're confused that means you either if they've done a bad job or they made a mistake or that you haven't under stood something if you can't understand the sentence try to read on maybe it's clarified later and then you know go back but again do not do not like just start a lot of times when I read paper previously like I wouldn't


understand something quite well yet and then I would be like oh yeah yeah yeah and then I noticed that I start skipping and skimming more and more because that would you know pop up again and again and I wouldn't understand it again and again and then at the end I would just be kind of glancing at the paper and I don't want to do that right here so I want to read every sentence and understand it okay so here then I find out about the loss and then I if I don't know something here then I'll go and look it up on maybe on Wikipedia or


something like this now I don't need to understand every single part of it right that's maybe I should correct myself so for example this bounding box loss here they talk about the second part of the max across the Hungarian Falls is this box loss that scores bounding boxes unlike many detectors that do box prediction with some Englishmen the other yada-yada they say the most commonly used l1 loss will have different scales for a small so here they basically talk about how they mix the losses they say overall our box loss


is that defined as this and this now I haven't I don't know what these losses are I just assume there are some bounding box losses so when I it's not true when I say understand everything understand the things that are integral to the story of the paper right how exactly they compute bounding box losses at this point I don't care I just assume that there is some loss that I can back propagate right I what is important is that they do this Hungarian matching thing right as soon as I get that I'm like ah that was this you know this um


this thing no this thing up here this thing this with the matching thing now I get it now I know uh there are always the same amount of boxes here there are always the same amount of labels here and all we need to do is somehow match them and I immediately think why is that relevant oh because when something is already matched to an object some other thing cannot be matched to the same object and that's how we you know prevent the fact that all the things predict the same thing right and so that immediately becomes clear and as I said there is


usually like one or two ideas in a paper I don't assume or I don't care what their exact loss function is because I've sort of gotten the idea up here of what the loss is about alright so I hope that's clear under very closely read the things and understand the things that are necessary for the story if you find if you think something's not necessary for the story and then later end up not understanding that maybe come back and you know read it again in any case I would I would rather I would rather skip something and assume it's not necessary


if I think so and then come back then trying to understand every everything but the things I do read I try to understand thoroughly okay then there's the architecture okay and that again I read closely and get backbone ok transformer encoder ok and now I understand much more closely a decoder ok and here I get now finally I get what this is about because n objects in parallel yada yada yada these input embeddings are learned positional encodings that we refer to as object queries and similarly to the encode we


add them to the input at each attention layer so now they name i've already seen these object queries here and the only word i actually need from this sentence are learnt the fact that their positional encodings I just kind of ignore as soon as they say learnt I know AHA these things here are learned they have actually they're always the same for each of the images they're just over all learned okay so now I feel I understand the entire model and yeah so then they say auxiliary decoding losses and this sometimes you have to pay


attention to like exhilarating because those are the things that here they say explicitly we found helpful to use auxiliary losses sometimes they they won't say why they did it they will just say our loss consists of three things and you know if you look at the three things only one of the things is really a part of their story so far and that you should immediately conclude that they've put in the other things because they tried it and it didn't work right so you can also kind of get an estimate of the brittleness and so on of the


system in that you see how many unnecessary things are there or how many things are not straightforward how many things aren't the easiest thing that you would do when you would go about and do what they did okay so then you this concludes this model or method usually the student section is called like method or model or something like this and you go to experiments now the main question I have so far or I have maybe I have some more questions about the model itself that I haven't been able to pick up from this section which is not the


case here but I simply keep those questions in mind and see whether they are resolved later right so I keep an awareness of what I don't understand but from here on my main issue is are they demonstrating that their story works right so they're here they're they're proposing a loss and a model and in my mind they now need to convince me that that works and that's that's it's not as easy as simply to show me some numbers that they are good at some benchmark they need to show me that they get those numbers because of what they claim so


here they claim well okay they propose a new they propose a new architecture so what they need to convince me of is that the architecture itself makes sense right but in other papers when when you propose like and when you say like oh we for example in an L STM when you build in an attention mechanism and you claim oh we you know the attention mechanism can look back at the source sequence in one step then you need to convince me that that actually happens right so you need to not only need you need to perform well you need to convince me


that you perform well because of what you claim your model does right so and that's often difficult and I specifically look out in the experiments for usually the question is like where are they trying to me right where are they trying to are or are they trying to me are they trying to cover up the fact that something doesn't work now all the experiments are always in the best light possible of course and you have to keep that in mind but a lot of times you can also already see from the experiments that okay are they doing


something weird are they not showing me some obvious experiment or and that's a lot of time because is there an easier explanation for why they get the results that they get other than their explanation right and it is it is their job to convince you that their explanation is the correct one for these numbers and especially if there is an easier one that they haven't excluded and then I don't believe the experiments if that's the case right if there is an easier explanation for the effect I'm I'm very skeptical but some papers have


an easier job here than other papers so in this paper they basically show results on a on a on a task and since their paper is about hey our pipeline is just easier than other pipelines what they first of all need to do is they need to like match the numbers of other pipelines and here I see that okay in these results often in the aftermath maybe a table or something here you see like this their model other models and their model is the best model in a lot of cases now if the best thing is of course if is if their model throughout


is the best the worst thing is if it's like scattered like this even if their model is the best but in every single benchmark a different configuration of their model is the best that's that's sort of a bad sign unless they can explicitly explain why that is and it's also not that good of a sign if these things are spread out like this like sometimes this baseline is good sometimes their model is better and so on so pay attention to that now in this paper it doesn't matter so much that's actually fine because what they're


trying to show is that their model is on par and way easier and they've already made the case in what way it is easier it's easier in terms of architecture if there were to say it's much faster then after that I would expect you know an experiment in speed while these numbers are matched so but since they say it's easier I've already seen the architecture I'm convinced of that now that they show okay our numbers match actually I'm surprised they even outperform a lot of times then I'm quite happy with these experiments so also


look for differences between numbers and the spread of numbers now it's not easy to say what if like point 1 is a bigger a small difference that depends on the task but if you know pay attention to these things pay attention to the fact that these results are noisy and oftentimes there is a lot more hyper parameter tuning going into the model of the paper then into the baseline model so why do you want to make your look your stuff look as good as possible and here is a little bit where the institutional credibility of someone


like Facebook comes in in that I tend to believe their results a bit more than other results megha but a bit more yeah also look at patterns that they don't point out in the text so if there is like a pattern if you see like an interaction between the number of parameters and the score or something like this just try to be on the lookout of that and see if you can spot something that you think or think about whether that makes sense or not in what your hypothesis would be so here we go on and okay then they go into


ablations and a lot of a lot of these papers do appellations and i generally appreciate that so here they visualize that the attention mechanism in their model actually refers to different instances right encoder self attention for a set of reference points the encoder is able to separate individual instances and you can see that pretty clearly right here where and even here with the overlapping cows and this is the sort of experiment that I would expect that actually convinces me that their architecture does what it says


that it does right and something like this where you see like totally overlapping things with the attention of the individual things visualized so telling me like especially this one right here the the foot of the back elephant actually being focused by the attention of the bounding box of the back elephant that's the sort of experiment that convinces me that their claims like that their numbers really come from what they claim it comes from okay so at the end of the experimental section you should always ask yourself


have they really convinced me that their story is true right that the improvement or winnig whenever they get an improvement or whatever they get whether is is due to the story that they want to sell me or could there be like an easier explanation or does something not fit is like other are the experiments different than from what you would SPECT here okay so these are these are my main questions are they are they convincing me of their story it's not do they have state of the art numbers I don't care I don't care even though like


sometimes so there is a bit of a catch I I don't care about state of the art numbers now let's say you have a table like this and you have a computer vision model and one of the models is like on the C for 10 data set now if your baseline model has like a ninety one ninety two percent accuracy on C for ten when I know the state of the art is 96 I don't care right I know like I've done C for ten I know with like I don't know five six layer CNN you can reach these 91 92 93 % accuracy and to get to the 96 97 you would actually be like in the


region of a wide ResNet and whatnot so I you know I know that even though you're a few points behind state of the art I know you know this this is valid still so I don't care but if you were to be like at 80 percent accuracy on C for 10 then I then I get a bit like I like it's pretty easy to get to 90 percent plus with like a standard CNN so there I immediately start to wonder why is there an explanation now this could be like a theoretical paper that says oh we investigate MLPs and that's why we only get that number so that's that would be


fine but if something is out of the ordinary like this then I pay attention but never because something isn't like the latest and greatest state of the earth that's just dumb ok and also if only evaluate what the paper claims it does right if the paper says we want to show that we are on par with current models then don't be mad if the paper doesn't outperform these models they didn't claim that right so yeah so after these ablations I'm actually pretty happy right here with the results and this right here when I saw this I didn't


I didn't expect that but I read the experiment description that these are these different learned object queries and what they do and that gave me an increased understanding of how these object queries actually work right so at that point I still had like a vague I knew that these are learned but reading this and sort of looking at it studying it a bit I was like oh okay then I understood even better what they are so again when I say understand everything in the method section you can still have questions and but you just have to keep


it in mind for later and then here I go on and there's this de TR for Panoptix segmentation and they here they propose like a new model so I first look at it and I'm like okay they proposed a new model they can do stuff like this now this is not object detection and again I'm not sure is this like a is this like a an add-on to the method or is was was this up here just an intermediate step to this and honestly after reading that I still wasn't sure it seems like something in between of course the paper is also a bit longer than other papers


it just it seems it's too long for just being a side note but it's too short for being its own thing so that was just a bit weird and I treated it as just like a oh we can also do this with our model but I didn't pay like too much attention to that okay so at the end I you know look at conclusions now the conclusions of a paper are much much often they are not nearly as informative as the introduction the conclusions they all often tend to be very generic and kind of hedging a bit against criticisms saying what would be up for future work


which is again hedging against criticism because you could simply say well we didn't do this that's future work yes so again I read it but I don't really pay attention to it and then I gloss over the abstract I just would kind of scroll through the abstract if there's something that catches my eye I would look at it and if not then not and then I basically go to the start and whenever I didn't understand something I go back I look at it again and I try to think are all my questions answered and have they sufficiently convinced me that


their story is the thing that really has the effect right here and then if I now were to make a video of this I've often found it useful to just put the paper away for a while and it's I usually get the best result when I read the paper the day before and then make a video the day after or if not I'll just you know put it away do something else do some email responding programming going outside eating lunch just some kind of a break between first read or between your first couple of reads and just I don't even think about the paper I just it's


kind of it's in the subconscious it kind of bruised right and I happen to think about the paper every now and then but I don't make a conscious effort to be like oh how am I gonna explain this and so on but I just found the the worst videos are the ones where I immediately make the video after reading a paper and I've just discovered that if I kind of take a break and then I look at it again right I look I don't read it fully again but I if I have if I have the feeling I've understood it I don't read it fully


again but I just kind of look at it and go again through this story and I think that's even if you you know wanna if you want to talk about a paper in a reading group or tell you know explain it to your friends or whatnot this is often very useful just put it away for a while let it Mellow and I find that helps a lot okay that was my process of reading this particular paper now we again this this is a high quality paper so it's I find it's a pretty easy read in that I simply need to understand what they did and I'm


I got you know basically my spacian is to kind of get to the next step in AI machine running etc and the what we what we see today is is a huge amount of success in machine learning but the sample efficiency of all of the techniques that we use today are much much worse than everything we observe in humans and animals in other words it take many more samples or many more trials in the case reinforcement learning for a machine to learn anything compared to humans and animals so a lot of people are very quick to draw

Yann LeCun: "Energy-Based Self-Supervised Learning"