algorithmically you compute it for all X's you compute that top part in the fraction and after you've done that you know it has to be rescaling you just compute the rescaling and multiply with that in the paralytic robotics book notation it's a little bit funny that sometimes they use the ADA as the thing you multiply with sometimes I think you divide by just gonna I guess be it be


okay with that it has different meanings in different contexts but ultimately it's just a rescaling and some people like to divide by the risk and some people like to multiply with one over that thing all the same law of total probability with integrals we saw that basil with conditioning we saw that we saw conditional independence what it means again it's an assumption we often make

Lecture 11 Probability Review, Bayes Filters, Gaussians -- CS287-FA19 Advanced Robotics


that will hopefully be approximately true or maybe even fully true so we can use it to simplify our math here's a simple example so we have a robot trying to see if the doors open or not okay what's the probability of doors open given some measurement of the sensor of the robot it is to illustrate that kind of reasoning behind why we often use Bayes rule probability


of being open given some measurement is a diagnostic distribution it's kind we're trying to diagnose what's the state given a measurement but those distributions often we cannot build we cannot deliver a sensor with that distribution because I mean you can't do that imagine you deliver the sensor to somebody and they lock the door and nobody has the key anymore then your


sensor can't know that because you didn't know that ahead of time what you really can build is the opposite distribution that causal one given a situation what am I going to measure and so that's what's be available to us and that's what we have to use but it's not what we want the reason about the world so we'll use Bayes rule to get what we want which is the opposite thing going from the


distribution that we know how to model which is the measurement given the state to the distribution we want to take action which is one know whether the door is open or not or at least a distribution over that for the causal one it's really just a matter of counting frequencies you might have a physical model that makes it easy but often you just kind of just do a bunch


of measurements so you see how often does a sense to read this or that as a function of whether the door is open or not and now I have my model let's do a concrete example we have we need to put some numbers into it probability of a measurement Z given the doors open is 0.6 Zija the door is not open is 0.3 so we see that when the door is open we are more likely to measure Z as the outcome


than when the door is not open so if we measure Z which should increase our probability that the door is open and our prior is 50% well we can apply Bayes rule we measured Z we can look at Bayes rule fill this in do the math which is described to find indeed that now after measuring Z which favors the door being open we have more specifically 0.67 probability that the door is open now


you could have another measurement imagine of another sensor wait wait go yeah let matching we have another sensor z2 how can we integrate that well we can again apply Bayes rule we already know that we can apply Bayes rule again even with conditioning in the back we already conditioned on one measurement just in a condition on the next measurement we actually do this as many times as we


want so the condition on many measurements we can do it one at a time and build it up so here we have the often a assumption is applied to simplify the math is that the sensor measurements are all independent given the state of the world because the state of the world is causing what you measure and so if the state of the world is a known entity then each outcome is


independent given that state that will simplify the math because then when we look at Z M the enth sensory measurement given the state of the world X and all the other sensory measurements we don't need to build a model for Z and given X and the other measurements we just need to build a model for Z and given X the state of the world which is exactly the kind of sensory model that we like to


build so we get a simplification happening over here we had ZN conditioned on everything that's the standard Bayes rule with conditioning but then we made an assumption there's not generality but we made an assumption that Z and is independent of the other Z's if we already know the state X and then this here has simplified then again the bottom we don't worry about there's


no X in it we can compute the top part for every value of x once we're done we sum it all together to know what the bottom is that will normalize it we can apply this repeatedly that the idea we just saw that this becomes Z on given X px given all the previous ones there's nothing specific about Z N and we'll see then we get a product over all Z is given x times prior over X and so now


all we need is that sensor model for each sensor given X we multiply probabilities together and we get our posterior distribution okay for the door example we had in this case a new sensor is z2 and when we see z2 it actually is more likely to be seen when the door is not open we observe z2 so what do you expect to happen mathematically we had a 2/3 probability


of the door being open ahead of time after sensor measurement 1 we observe z2 that probability should drop but let's check we can do the math we just apply Bayes rule fill in the numbers and indeed the probability drops as we expected all right now there is one thing I want to point out here is that a typical pitfall is that real-world sensors are not always super independent


so you might have a sensor and you might say okay maybe I have a laser rangefinder and it reads a reading but then maybe send out two beams next to each other but those two beings might encounter the same kind of noise or the same kind of interference in their paths and so those two measurements might not be independent and the assumption we just went through where there's all


simplifies just becomes a product of things is violated even though often in the math will want this and practice be careful you need to think carefully are these sensory readings really independent or imagine the laser rangefinders reading something at time T and you run it again at time T plus 1 but nothing has changed in the world then are those really independent


readings not totally necessarily because you're measuring the exact same configuration with the exact same sensors so maybe it's not completely independent so be careful what can the effect be imagine you imagine you think they're independent these readings and they're not what do you think will happen let's think about this imagine we have really one reading and really what we


should be having in this equation because there's only one in one measurement happening all the others are just copies that are not independent information they're just copies of that one reading that's how dependent they are extreme case of dependence we're just you know instead of fitting it in once into our Bayes rule thing we say hey why don't we feed it in ten times


because that way we can incorporate the information even more all right what would happen you feed it in ten times you multiply this thing in ten times so what will happen is that whatever that measurement is favoring you'll become overconfident in it if your favors door open and maybe normally with one time incorporating that measurement you go from 1/2 to 2/3 if you cooperated a


hundred times you multiply it in hundred times all of a sudden you'll be at 99% probability instead of where you should be which is 2/3 that's exactly what happens if we look at the graph here so here is a graph showing as a number on the horizontal axis number of times the same measurement is incorporated using the math that is the assumption that these measurements are independent and


we see that very quickly you can actually flip the probability from being 99% being one observation one state in the world to the other state in the world X 1 versus X 2 so be very careful about that another places might come is like you might maybe have I don't know accelerometer or gyro and the gyro is supposed to measure independently the angular rate of your system but the gyro


is a physical system under the hood or something physical happening there and so it might take time for that physical thing to settle in and really get that measurement and if you you know read it off I don't know ten thousand times per second that might not be valid that might only be 10 or 100 independent readings really and if you then use the 10,000 readings instead of just the ones


that are actually independent you'll become wildly overconfident in whatever it is that you're measuring rather than keeping a reasonable distribution of our possible states of the world okay let's take a couple minute break here and then let's look at Bayes filters and gaussians actually much more I did it's not as unstable for alright let's restart any questions


about the first half yes so I've in a lot of things we assume a first-order Markovian system and kind of touching on the example that you were talking about at the end have you seen in practice whether maybe the first-order Markovian assumption isn't appropriate but a certain like n step Markovian assumption is appropriate so that we can find this balance between


having something that we can get accurate I guess quantification from but also the joint distributions are not intractable yeah so it's always a trade-off between tractability and match with the real world I would say I think in general first order assumption I mean in principle if you have full state then of course definition of full state means that the first-order assumption will be


true otherwise it's not full state but in practice your state will not be maybe the full state of the world to be a approximation of the state that you use as your representation for state space and as a consequence first sort of assumptions might not exactly be true I think often it's dealt with in pretty much ad hoc way where people just look at the system and say ok for this system


it seems like we need to look this far back or that far back to get what is effectively the state of the system by including enough history but I haven't seen like if necessarily like a very clear-cut way to define it in general and this ties directly on what we're going to do now which is Bayes filters so often the world is dynamic because robot does things other agents do things


time is passing by the world changes in various ways as a consequence so how to incorporate actions into what we're doing as well as our sensor readings so typical actions robot turns its wheels to move robot uses manipulator to grasp an object a plant grows over time and takes up more space actions are never carried out with absolute certainty so in contrast to measurements actually


actions generally increase the uncertainty dibal you have is you will have measurements reduce yesterday in your distribution then action gets taken which introduces new uncertainty and the distribution becomes higher variance how to model actions well we can again do it with a probability distribution so we can say the prohibition of our next state X prime given current state X and


action U and that again is a causal model it's something we can build a model for and then use to calculate the distribution over next States so for example a robot might try to open a door and it might succeed or might not succeed you might have a model for that so you might say if the doors already open and or in this case closed the robot tries to close the door doors


already open well 10% chance it'll just stay open because it fails 90% chance it managed to close it if it's already closed and it will keep it closed okay that's a model that might or might not be correct but these are the kind of models that we're going to be working with such that we can do calculations about possible states of the world based on our actions we took and sensory


readings we get well this is exactly the law of total probability we're applying here the conditional distribution for X prime next day given the action U is well we have a distribution for current state that's what we assume we assume that we know the solution for current state will then multiply in X prime given U and X to get to joint over X Prime and X once


if they joined over X Prime and X we actually just want X Prime so we sum out or integrate out X to just get the thing for X Prime that's exactly what's happening over here loft all the probability with some conditioning so we can run this for the robot example and I'm going to step through the details here but you can do you know with those numbers we have the


math you can see okay given the action that we took the probability of clothes become 15 out of 16 and the priority of open is 1 out of 16 how about measurements measurements are what we've been talking about the most so far we'll use Bayes rule Y again because we have easy time getting a model for sensitive reading given State but we don't know how to get a model for


state given sensory reading directly because we need to know the prior over States and wanted to use Bayes rule to get that now in Bayes filters the framework is that we get a stream of actions and observations we have a sensory model a action model or a dynamics model and a prior distribution for where the state of the system is starting from so these are our Givens


what we want is an estimate of the state of the system at all times so we want to keep a running estimate over distribution over possible States for each time T pictorially often it's drawn this way this graph captures the causal relationships that are present in our set of assumptions so X again is the state over time use the actions we take actions will affect the next state the


previous state will affect the next state and the observation in this case assumed to just depend on the state the state of the world determines what we might measure and so when we get to look at this graph the explicit assumptions being made is that the measurement at time T given everything that happened before everything happened before only depends on X T so for no X T no past


states matter when we know X denote past measurements matter no past control inputs matter it's all summarized in state XT we're going back to the question of course if your state space doesn't capture everything about the world then this will not be true in this assumption will be violated and typically that's the case you never captured the whole world stayed in what


you do but hopefully you're close enough that this actually works then same thing for dynamics next state could in principle depend on everything that came before but the assumption we make is that it only depends on previous state and action taken again the paralytic robotics book makes some kind of different kind of indexing that I'm personally used to but since that's the


the best book to go read probably on this if you want to read more I'm sticking with their annotation so they said the action taken UT is the one you take that will land you in XT normally you would call that UT minus 1 I think that's what almost everybody calls it but in the book they call a UT it's the action that you know you take right before you land in state X T so that's


what it is and that's also reflected in the graph over there UT it's actually you take right before you end up in state X D so unlined assumptions are independent noise meaning that the this thing here these two assumptions means that if you let's say how did the mystic model plus noise that the plus noise effect is independent at different times and it also assumes if you are if you're


going to say whatever I get out of these calculations is the correct distribution that's only true if these assumptions are correct if you make any approximations in your dynamics model in your sensory model well those approximations will result in miss estimates that you end up with but it's the best you can do but keep that in mind it's there's a lot of assumptions


made here and so you only get the result relative to the quality of the assumptions that you made the quality of the models that you built in okay let's step through Bayes filters so what builds up step-by-step is the probably the key thing for the second part of lecture so we want to believe which is a distribution over X D given everything from the past including also the current


observation of time T now remember it's going to be a recursive thing we're going to already have this presumably for time T minus one and we're going to see if we already had it for t minus one how do we get it for time T what's the latest thing that happened latest thing that happened in that progression is the measurement ZT how do you incorporate a measurement Bayes rule so ignore


everything but XD and ZT what we do p XD given ZT we'd say Bayes rule its ZT given XT times XT okay that's exactly what we're doing we're going to apply Bayes rule and then some normalization right but remember there's also that other stuff and we'll just carry it around everywhere we know that we can carry things around everywhere as long as we do it consistently so it's


just Bayes rule apply now we had some assumptions that the measurement at time T given XT does not depend on anything else so we just simplify accordingly okay so we've not yet gone from XT minus 1 to XT we've incorporated the last step the measurement can we now do the transition from XT minus 1 to XT well let's think about it how do we go from XT minus 1 to


XT well xt- one I've already have it we're multiplying the dynamics model to get the joint and then sum out XT minus one okay that's exactly what we're going to do we now say the distribution for XT is XT given XT minus 1 times distribution for XT minus 1 and integrators sum out XT minus 1 and sure we're carrying around all this extra stuff here you want or UT


and the Z's and so forth but we know that's just extra stuff we can care around consistently this is just the law of total probability for XT and XT minus 1 then we say well can we simplify this because our mark of assumptions yes we can we don't need to conditioned on everything that's conditioned on there XT given everything in the past only depends on UT and XT minus 1 and this


here is now our complete recursion because this is a thing we assume we already have is the same thing we wanted to find for XD we have it for XT minus 1 and now we can just repeat all the way back to X 0 and we're good to go so at every time step as we track our system we can incorporate the controls we applied U at time T here gives the tribution over XT this thing over here


gives distribution over XT given that what we already knew about XT minus 1 and the controls we applied and then we get a measurement we're multiplying the likelihood of the measurement and then we were renormalize and if our distribution for XT and we can keep tracking over time all right the mark of the assumption makes this even slightly simpler notation and we're


good to go all right so and then the last thing here is just the notation the book uses BL belief of XT minus 1 is a shorthand where there's no there's no assumption here it's not like we can delete all this stuff here just to be clear this stuff is now being deleted by some assumption is just the notation beol XT minus 1 by definition means XT minus 1 given all this is just


a shorthand notation so for the second-to-last equation or like the third line from the bottom and the second line it goes UT at the last one and then it's CT minus one what happened there like on the conditioning oh ok the only thing that happened there essentially that the UT was removed yeah it's not fully spelled out but essentially UT does not play a role for


XT minus 1 because it comes after and so UT was removed from that that's all that happened and so this is the final result we have we can put this in an algorithm and this the algorithm that will be running and this is the version for discrete States of course where we can just track the Blee compute this was actually very simple you start out with some distribution prior over X your


distribution over X then when you get an observation you're multiplying the likelihood of the observation given X of course if you do that for all possible values of X you sum that together normalized by it you every new believe for X after that observation has come in if it's not an observation that came in but you took an action well then you use law of total probability then a much


smaller gets multiplied in you get the next state and so you can run they don't if you alternate between Z and you you can just have a bunch of Z's coming in incorporate them then take up some actions incorporate that and just repeat over time whenever something come in act upon what happened so in summary Bayes rule allows us to compute probabilities are hard to assess


otherwise that's really what's going on here under the hood it's that we don't have direct access to the distribution we want we've access to it sometimes the reverse distribution we can use to get what we want under the markets assumption this recursive Bayesian updating can be done very efficiently and Bayes photos are kind of the common pricing tool for estimating the state of dynamic systems


here's a simple example imagine your robots supposed to localize has four measurements and that means measures in all four directions and it checks whether it's a wall or not in each of those directions it's a noisy sensor and we're going to not go through the math here we're going to qualitative let's see what happens well if it's if it's really over there but we of course don't


know where it is the gray e the uniform gray means our distribution is uniform over all possible locations we show the location of the robot just to you know know what's going on but the robot doesn't know that robot just could be anywhere get some measurement to come in since wow I only see things above and below me not to the sides so the Bayes rule update will result in


the new distribution then they might say I'm going to move but it might have noisy motion so it then kind of spreads out the distribution a little bit then it measures again which shrinks it again and this process repeats over time and over time it might localize itself in this building of course this assumes that access to a map of the building otherwise this whole


base update rule wouldn't work out in this case okay that's it for Bayes filters I want to now move on to Gaussian so we'll look at univariate gaussians multivariate gaussians we'll look at law of total probability for gaussians which already covered for discrete case we'll look at what does the math look like for Gaussian so we'll do that next lecture but that's on the


menu for us for gaussians and we'll look at conditioning Bayes rule how it shapes up for gaussians univariate gaussians what did it look like is this kind of blob kind of distribution where most of the mass is in the middle but it can actually go all the way to infinity on both sides there's something called standard deviation Sigma and 68% of the mass lies within one standard deviation


of the mean and then it kind of decays out from there the mean is usually denoted by mu sanitation by Sigma and then the variance is Sigma squared the density itself is this thing over here so it's exponentiated negative x minus mu square so let's think about that when X is close to MU and the X is equal to MU that's zero and you have e to the power power zero which will give you one


and then if you are the further you go away actually could be even let's see yeah each of our zero would give you one if you go further away from you you start moving away from you this thing will become and the exponent will become a more and more negative number and that will make the expo that negative number will be a lower number and density drops it's exponential Exponential's drop very


quickly that so why this nicely integrates to one still because Exponential's drop quickly enough there's not a lot of probability mass out in the in the faraway parts the normalization constant upfront that's just you know you do some calculus and you find out that to make sure that this quantity here to make sure it integrates the one you need to put upfront 1 over


Sigma of 1 over square root 2 pi ok so what are some properties of gaussians well this is integrate to 1 so we know that this thing integrates to 1 the expected value of your variable X is expected value is you integrate over all possible X can take times probability of taking that value if you work out the integral and not proving that this is the integral here I'm just saying if you


work out this integral you'll see the result is mu so the expected value under a Gaussian is immune not to an expected either because I mean in the name is also saying it's the mean but you can actually do the math and principle to find out that mu is indeed the expected value under a Gaussian distribution the variance is defined as X minus mu squared on average what's your square deviation


away from the mean you can do the math compute this integral you'll see Sigma squared is the variance which is what we already called it but in principle you know you have to calculate it verify that the name we use for Sigma squared is actually really what variance means and indeed it is now you might say why might we care about gaussians these integrals I mean we figured them out or


somebody figured it about but they're not necessarily easiest but at least they can be done well there's another reason to care about them aside from them being convenient to work with the central limit theorem classical CLT says that let x1 x2 be an infinite sequence of independent random variables with an expected values mu and variance Sigma squared then we define a new variable ZM


Zn is the sum of all of them minus the sum of the averages so this should be centered around 0 because the sum of all the mind is the sum of the averages or expectations and then normalize by this quantity over here it's a scale down then for the limit of n going to infinity with infinitely many of those variables being put together in a sum and send it around


zero and normalized then we have Z going to a zero one so zero mean Santa Aviation one Gaussian so what that means is that if whatever you care about is the effect of multiple independent factors the resultant variable is often distributed more or less like a Gaussian and the more independent factors contribute to the variable you only care about the clothes that variable we


ultimately care about will be distributed according to a Gaussian so it's not just about convenience of math though that's a big part of it another big factor why gaussians are a reasonable thing to use is that in fact even if there's enough underlying factors it might really be a Gaussian the thing you're looking at about multivariate gaussians well here's what


it looks like these are densities where X is now a vector so we again see a normalization happening up front which is has a determinant of Sigma in it Sigma is a symmetric matrix the covariance matrix and have X minus mu X minus mu so whenever you see something like this often it's a little hard to make sense of it directly if you haven't seen it before your simplest thing to


you say okay yeah there is matrices and vectors but whatever is just scalar if it's just scale it looks like a single univariate gaussians again you just have X minus mu squared if it's not a scalar what do we have well Sigma is a symmetric matrix so the inverse of Sigma is also symmetric matrix so the symmetric matrix symmetric matrices are as we know from we've covered lqr just


the rotation away from being diagonal matrices just a coordinate transformation away from being diagonal so we can just as well think of it as diagonal matrices if we were just working the correct coordinate system okay so imagine it's diagonal then we really see like x1 and mu1 attracting with the first entry in that diagonal and x2 mutant with the second entry and completely


independent no interaction so x1 and x2 are their own gaussians in that coordinate system where now they might have different variances because the dialogue could have different entries for x1 and x2 but Elmi is just saying oh we have to gosh is 1 4 X 1 1 4 x2 they could have different variances that's it now if we're not on the coordinate line system shirt then it won't look like a diagonal


but the intuition remains the same there exists a coordinate system where this would be diagonal and easy to think about I also remember back when we did lqr I told you any matrix can be any symmetric matrix in a quadratic form is fully general because we have a non symmetric matrix the non symmetric part cancels out anyway now we did some math on the board and that feel free to revisit


there's no reason to ever consider a non symmetric matrix Sigma because the non symmetric part will just cancel out in that quadratic form alright so we can compute expectations here to expected value of variable X is of course is mu expected deviation of X from the mean is actually entries in Sigma so this is saying expected value of x I minus mu I so how much bigger is the Exide than its


mean x how much bigger is xj then its mean well if they're both together bigger and together smaller then this will always be a positive quantity and will have a positive number come out if one of them tends to be bigger when the other one is smaller and the other way around then they'll have opposite signs a negative entry will come out and they're completely independent and their


bigger is smaller then I'll cancel it be 0 so Sigma IJ is positive when they both together tend to be above average negative when they have counter correlation in terms of above below average and 0 when there's no relationship between when they're above or below average let's look at some examples what this looks like so here is a plot of a density of a Gaussian mean


at 1 0 the first axis is this 1 the second axis is this 1 in all these plots Center deviation 1 for each coordinate so symmetric Gaussian which keeps television while we shift it now the mean is at negative 0.5 for the first query we see it shifted what if we shifted even more will so shifted for the second coordinate to mean again this bump just moves around let's look at


some more examples again a starting point a 0 centered variance 1 Gaussian we then reduce entries on Sigma so Sigma is diagonal here setting stay acts as a line nicely symmetric point six point six it becomes a taller peak because all the density has to be closer because you're not allowed to be as far away from the mean as often so the mouse has to be more centered you can make the


standard mission larger and things will spread out you can also do things that are not diagonal matrices so here again on the left the standard one in the middle we said x1 and x2 are positively correlated so what do we expect we expect that there is a lot of mass along the axis where x1 and x2 are both above the mean or both below the mean the mean here is zero so along the main diagonal


we expect there to be a lot of mass which is exactly what we see then here we made it even more 0.8 and we'll get even more mass along that main diagonal because whenever one is above average the other one should also be above average below average the other one should be below average most of the time you can draw this all so differently so drawing in this kind of 3d like sketches


is sometimes hard to do much easier to draw contours so the contours of the densities shown below the kind of 3d plot then here's a corresponding density contour for the middle one and for the last one you see indeed the ellipse runs along that main diagonal how about some other examples here is negative correlation so here it runs along the opposite axis when x1 is above average


x2 is below average and so we get it to run the opposite way we can make it even more negative negative point-eight even stronger negative correlation or we can make it positive again and also make one have higher variance than the other one so the first coordinate has more variance than the other one any questions about these okay then next lecture we'll do the math


for the total probability law and Bayes rule for gaussians actually one quick extra announcement for today's today's lecture is probably one lecture where the pace is probably the most off for most people because you all have very different backgrounds so if it was too fast for you make sure to review everything really carefully to be ready for the next few lectures all three


howdy-ho how's it going so today we are going to try out the DET är the end to end object detection with transformers from facebook AI research and they have a github repo and they pretty much give you everything like the model the pre trained weights and so on so today we're going to check out how easy it is to get started with that so in order to do that they have like a collab but we we won't

[Code] How to use Facebook's DETR object detection algorithm in Python (Full Tutorial)


look at it too much I've glanced at it and we'll basically see how far can we go without looking at it too much and how easy is that so what I've done is I've spun up a collab that I will share at the end and I've imported torch and just loaded the model so you don't have to wait for that to happen so I've loaded that up and now we have it in the cache so now we can


basically go ahead and load an image into the model and try to detect objects in the image so first of all this is super easy right you simply load this from torch hub it's kind of like the the tensorflow hub you simply give the name of the model you say I want the pre trained please chugga-boom you now have a model so if we look at that model this is going to be this entire Det our model


right here with all the transformer and ResNet and whatnot okay this is almost a bit too much right here so what we want is an image so let's go find an image where better they find an image than Google so let's find an image of dogs because dogs is one of the classes in this cocoa dataset this one's nice right okay so we want the image address we want to load it in here somehow so so


that the URL is let's make this into some sort of like an input thing where we can paste the URL right here okay there we go so we have this right here and that's the URL all right no that's not the URL at all is it but a beam but a boom what about cool better now we need to load this for that we gonna use the requests library always a pleasure requests requests so the way to load a


binary file is you can put the URL here and you can say streamed here I glanced this from the other thing and the raw entry will get you the eventual deeply bytes no oh sorry get darrell streamed stream yeah so this will get you the sort of the the bytes of the image and then use just say image dot open and of course we need the image from a the pill library the python image


library so import image we got that and we can open that image up and with a bit of luck yeah yeah so this model expects I think cocoa dataset is 640 by 480 images but they if you can see right here and we're gonna take a quick glance at their transforming they resize it to 800 so we're gonna we're gonna steal that part right here people last time where some some found it really funny


that I called copy pasting to go Suraj so will from now and we'll call it just Suraj Inge what we also need are the class labels because that's in defined in the cocoa dataset right so these are the class labels let's take those and okay so this T here these are torch vision transforms we're gonna need that so from say so if you don't know torch vision it's kind of an addition to PI torch that


just helps you with with images and has a lot of data sets and these transforms they're really helpful because so let's call this image because you can you know resize but they have much more like random cropping and rotating images and so on pretty much everything you need for pre-training and this here is just the standard image net I believe the image net normalization so these are the


means and these are the standard deviations from the image net data set and let's already resize our image actually to this weight hundred and I believe I believe if you rescale the 640 to 800 you get 600 here right fairly sure okay and then let's display it just because we can okay what it's it's a bit squished but we don't care and let's put that up here so we only need to execute


it once nice okay so from now on it should be a breeze so what these transforms do is they resize the image okay we don't need that anymore they make it into a tensor and then they normalize by that so if we run our image through this because our image right now is this is pill image right so our our image is this pill image but if we run it through the transforms then we'll get


a tensor so that's pretty cool so the model as it is a deep learning model that expects batches so we'll unscrew is that in the first dimension and then we get batches so shape let's see we don't have on skis no of course we don't so this is a one image of three channels of 600 by 800 so this is the Y index coordinates I guess are shifted yes in pi torch cool so we'll call this


our image tensor now we just need to put it into the model so model we put that in there and since we don't let's actually up here put the model in eval mode I don't know if that's already done but you know you can never be sure enough that the batch norms aren't so I think it probably doesn't have batch norms okay you're not utilizing the GPU we'll do that we'll do that


Thanks so how do we use the GPU we put our model on the GPU model equals model CUDA yes yes yes I think so this is gonna work okay we're gonna come back to this later so we forward our image of course we also need that on the GPU and it's worked did this work this worked nice okay and since this is just for evaluation we should probably go with no grad right here because we don't need


this whole gradient stuff if we do that okay I'm dumb there you go and nothing happens of course because we need to capture the output somehow let's look at that output Wow Wow just wow so the output is a dictionary right because we get back class labels and bounding boxes so let's look at the bread boxes let's look at that tensor that's a tensor very nice let's look at its shape let's not print


giant tensors anymore cool so since this was a batch of one we should probably go the zeroeth and you can see right here there is a hundred bounding boxes and each one has four numbers and if you go with the other thing that's in there the log it's then you'll see that there are also should be a hundred log it's and hello there should be a hundred log it's and each one is of size 92 because there


are 92 different classes 92 we'll see about that well one is going to be the nothing class right by the way how many classes do we have we have 91 classes okay cool we can deal with that all right so what are we gonna do next what we want to do is for each of the for each of the for each of the log it predictions we want to find which classic corresponds to so


what we're going to do is we're going to take the Arg max of the last dimension right so you can see here almost all of these things correspond to class 91 and class 91 is not in our classes because our class is only length 91 so that must be the nothing class so what we can technically do is for log its and boxes in let's just zip them together and [Music] like this okay class is oops


class as the law gets Arg max if that's 92 or let's say safe that's larger than the length of our classes we'll just skip it for now okay so that should work somehow and if not then our label should be the class index right here so let's just see what the detector detects right here it detects nothing why does it detect nothing that's isn't seem good what are we doing wrong we zip together


the log it's oh yeah of course we still need the zero with entry we are dumb dumb dumb cool so so so so we can delete this and now finally beautiful dogs - dogs detected excellent so now for each of these dogs we want the bounding box okay so now we somehow need to think of how are we gonna draw this on an image and well let's let's actually make a copy of that image because I don't really trust


myself and then at the end of this we're just going to display that image right now actually the reason I make a copy is because in these in this pillow library you can actually draw on these images and we're going to that to draw these bounding boxes so for that we need an image draw if I remember correctly and I think later we also want some text so we need an image font yes


all right so let's draw a bounding box right here where so first of all let's look at that bounding box let's call this box box print box dot shape and break right here what's happening let's not do this right now so this is a boxes of size four now this could be two things it could be X 0 y 0 X 1 Y 1 so the two corner points or the kind of the boundaries or it could be X Y width


height now from the paper I know that they predict the center and the width and the height so I'm gonna go with that and I'm just gonna guess that it's like X Y WH and not some other way around if this is a bad guess then yeah we'll see we can just print out one of these boxes and honestly that looks reason oh by the way we should scale that up yeah so these are normalized coordinates


probably between 0 and 1 so we should scale that up so we should probably the x coordinates which is scaled by 800 and the Y by 600 so let's do it so first of all we scale our box by 800 in the X and here is a Y and the width is the X direction and this is the Y Direction boom okay we should probably get that on CPU will just hack together a bunch of things right here ok so now this isn't


the correct so we sold our x and y and WH are going to be this box so now we need to actually draw on the image we're gonna do that so let's first go X 0 X 1 is X minus W 1/2 X plus W half y 0 y 1 is the same for a y with H plus H half Coolio now we need an image draw object so I think draw on this image so whatever you draw on the draw object will end up on


the image so we can use that to draw a bounding box and let's just quickly look it up so pill Python draw rectangle maybe there we go okay so there's this rectangle yeah there's the rectangle function and you can see you put in a shape XY here and width height like this wait for real we wouldn't even have to need to transform it I'm pretty sure you can go X I thought I remember you could


do the different thing as well but it's called rectangle okay so let's do that so draw rectangle and we'll go we'll go X 0 or we'll go X Y width height let's display that down here yeah that looks that looks nothing like we want but it's you know it's a start maybe actually we need the other thing here we need X 0 y 0 X 1 Y 1 mm yes yes doggy okay we still have the break in here


now we get both dogs nice nice okay let's do I think Phil yes red and let's go for with five or so five seems like a good width oh god five is a terrible with oh it's not feel I think it's its outline yeah yeah okay okay let's go still go with five cool we got our dogs now we need to put like some some snappy text labels I think there's actually a pill image draw


text I think that exists because I've this font thing yeah exactly so you need the font thing get it font in there and then yeah exactly you could put a text like this okay so you probably need the x and y coordinates of the text so let's do that W dot text and let's just go with x and y right here put it right in the middle and the text is going to be our label of


course and we want the fill that's now going to be the color of the text let's go with white and the font we're going to load some font right here font dot how we're doing this true type true type ah no not cheating let's just go with regular fonts it won't look as fancy but we'll be fine so we're where is our text you see it I don't see it red let's make it red yes there we go okay


so it wasn't red enough this should work on it so I did we just I just not see it I'm domina cool so we have two dogs how easy was that actually we wasted the most time with like bounding boxes and stuff absolutely cool right okay so now we can have some fun with it I'm going to scale this down for a bit because you don't need to see the actual code anymore so


much so you can see the image more so we'll go to the images and the first thing I want to do is the dress what does this think of the dress okay so we'll copy that and we'll go into our collab and just paste this right here butter boom but a beam sounds nice and what is wrong the size of a tensor must match the size of a tensor we do something wrong transform image or images this


maybe this is like an RGBA image I think if this is rgba we should just convert it to like an RGB pretty sure you can do something like this right here this should work as an alpha Channel then that will remove it yes now it works okay let's see what the model thinks of this yeah okay apparently there's a car and there's a surfboard and there's a person and there's a person nice see well we didn't


figure out whether the dress was blue or white through gold it was just a person now they you could actually like threshold by how sure you are of a given class but where's the fun in that so let's go further and let's do some Rorschach inkblots because those are always lots and lots of fun so which one should we go for it this one looks like fun okay so we'll put this into here and


it's astonishing right it's this cocoa data said it only has these 90 classes like it doesn't have anything anything else so it's a cake it's a cake and this here what is it okay we'll have to go maybe with blue what is it stop sign okay but so you might think it what if what if we want more like what if we want more predictions so there is a hack right right now the model can always


assign math to this not a class thing like right here this class 91 in order for it to say I don't think there's anything there but generally we have a hundred predictions right so you see where this is going so yes let's let's change it but let's change it up a bit and let's go here let's first extract these tensors and boxes okay so we have the boxes and this


and log its and boxes okay so we got that what we wanna do is basically we want to filter the we want to basically just remove the last class before we do the Arg max and thereby we want to force the model to make a prediction now it won't be a very good prediction because of course this is only the second highest class and it's arguable how much that counts but still it will do


something so this must be done in the log it's right so well look at the log it's and the log it's our of shape 100 so we have 100 predictions of 92 classes now the first thing we want to do is just remove the last class so let's go everything here until the last class all right so now we have 91 actually let's make it more generic whatever thing however many classes are okay so we


don't have this class anymore so now if we do the softmax over the last thing we can technically we get 91 but now they're normalized so they add up to one so it's kind of a probability distribution next we we want to find the max over this and that that will give us a max output so we don't want to plot all the 100 predictions because that would just be like like squares all over


the place and we'd have no clue what happening so this max output right here this what we're trying to find is we're trying to find a let's say the five best predictions or so the five ones where the model thinks where the model is most confident it's not really good metric but you know so these are the probability values of all of the hundred predictions so what we want is like the


top K okay so let's go with five and again we'll get like a top K output let's call that top K and I think it also has like values and indices yes so now we simply need to filter from the log it's and the boxes where these these top ones are so well filter the log it's [Music] will filter the log it's by that top K indices and we'll also filter thee I am not very gifted today boxes


by the way I'm using a collab just because it's nice to kind of play around with a model because if I were to use a file I'd have to restart reload the model over and over again just not as nice so now we have the log it's and the boxes and if we do that right now we get always the top 5 predictions how nice is that and you can see the top 5 predictions are probably still kkkkkk


cake and just to verify that and we can put its shape yeah see this is what I don't like about this stuff yes okay so we just have five predictions of 92 things and we don't want the 92 we've already said so we just want the 91 let's actually could put that here [Music] okay so now we have five by 91 and now to give us the top five are there we go so many takes and many stop sighs that's


fine that's cool so the ultimate test right here is going to be yes the human adversarial example let's check it out so we put in a Jackson Pollock image and we'll see what the model says now we're actually forcing it to make predictions right so it can't escape it will need to do something okay I made another mistake I would need to copy the image address right here like this


that's what happens when you're not an idiot you get the actual image so what does the model think of our pretty image okay can't even read that so let's make this into white bird bird bird okay lots of birds in this image clearly clearly lots of birds in this image let's try another one let's go with this this this one yes yes absolutely love it love it okay so we copy image


address and beam Mormons Wow there's a lot of birds in these Pollock images just so many birds okay let's try one last how about this one this one is a bit more human-friendly right put it in here and and and okay we get some detections there's a clock right here there is a what's that how's horses let's print let's print the labels so just so we know what they are


cake horse car horse and clock okay so I see the clock like this here is clearly a clock then this rectangle on the right side must be something let's put this to read as well now that's terrible ah white back to white how about back to white okay clock we got horse right here and house probably and the entire image is again a cake yes okay so as you can see it is a pretty pretty


good system but of course it is only these 90 classes but it's for now it's a it's pretty cool and it works pretty well and just the easiness with which you get which which you can get this stuff elephants in Kruger National Park just the easiness is astonishing you can just load it up kind of have this have a bit of a notebook and with a bit of like a very few lines of code you can put something


together that detects these bounding boxes lots of elephants and remember we only have the top five elephants right here so what happens if we go for more where is our top k so here we can let maybe say the top 15 predictions and as always if we want to make the model to make its own decision we can simply revert back and add back the no class label all right with that I hope you


Let's imagine for a few moments what our life would be like if we could access let's say 20% of our brain's capacity If you want to have something show up in your life The kind of person you would like to become manifest something new into your life something powerful, whatever it might be You obviously must first be able to imagine it Your imagination is able to do all that you ask in

YOU ARE THE CREATOR | Warning: This might shake up your belief system! Morgan Freeman and Wayne Dyer


proportion to the degree of your attention So what kind of attention do you place on your desires? Einstein's most famous quote one of his most famous observations. He said imagination is more important than knowledge Knowledge is limited Imagination encircles the world Logic will get you from A to B, but imagination will take you everywhere Make your future dream a


present fact by assuming the feeling of the wish fulfilled That which you feel yourself to be you are and You are given that which you are. So assume the feeling that would be yours were you already in possession of your wish and your wish must be realized so live in the feeling of being the to be and that you shall be if this assumption about what you would like to become is


persisted in until it becomes your dominant feeling the attainment of your ideal is Absolutely inevitable You must first assume the feeling of a wish fulfilled in all aspects of your life Don't allow anybody elses opinions Don't allow what it says on the internet. Don't allow the research. Don't allow what anybody out there tells you is possible or not possible


for you if you advance confidently in the direction of your own dreams and Endeavor to live the life, which you have imagined You will meet with a success unexpected in common hours it will chase after you if you can place into your Imagination what it is that you would like to attract and begin to feel it Start retraining your subconscious mind and your subconscious mind


it responds to what it is that you suggest to it The subconscious mind moves your life 96 to 97 percent of everything that you do is done as a result of your subconscious mind And when your subconscious mind gets programmed it goes ahead and Respond to whatever it is. Your conscious mind has placed into it You are the creator this is the mystery This is the great secret known by the seers and prophets and mystics throughout the ages. This is the truth that you can never know


intellectually Many of you as I have been as I am are where you are in your life Based upon what you believe and it's not just what you think you believe on the surface It's also your shadow beliefs that are holding you back from moving into the life that you believe You deserve What I know is if you're not looking at the shadows if you're not looking at what is


Subconsciously running through the tape in your mind telling yourself. You're not good enough. You're not worthy enough. You're not smart enough You're not enough which is a tape that's playing for a lot of people If you're not conscious of that then you end up acting out of that Belief system and not out of what you know to be the truest or want to be the choice for yourself


You are where you are today in part because of what you've been saying about yourself Words are like seeds when you speak something out. You give life to what you're saying if you continue to say it Eventually that can become a reality you Are planting seeds when you talk at some point you're going to eat that fruit. My challenge is make sure You're planning the right kind of seeds if you want apples


You have to sow apple seeds if you want oranges You can't plant cactus seeds poison ivy seeds mushroom seeds You're going to reap fruit from the exact seeds that you've been sowing in other words You can't talk negative and expect to live a positive life You can't talk defeat and expect to have victory you can't talk lack You're not enough can't afford it Never get ahead and expect to have abundance if you have a poor mouth. You're going to have a poor life


And this is great when we're saying things like I'm blessed I'm strong I will accomplish my dreams I'm coming out of there That's not just being positive. You are prophesying victory Prophesy and success prophesy new levels and your life will move in the direction of your words But too many people go around prophesy on just the opposite. I never get any good brights. I'll never get back in shape


Business is slow. I'll probably get laid off Flu season is here. I always get it. They don't realize they are prophesy and defeat It's just like they're calling in bad breaks mediocrity lack You don't become what you want because so much of wanting is about Living in the space of what you don't have that's why Jim Carrey's story is so powerful Because he started to act as though he already had it. He would go up to Mulholland Drive


He would drive away sing thinking. I already have those things I just haven't accessed them as yet I believe Those things are going to come to me and I'm going to act like they are so I'm gonna move forward in my life in order to Draw that to myself in such a way that my actions are in alignment with what I say, I believe So if you start to think about that really why are you where you are in your life the choices that you have made?


Have been because of what you believe to be true for yourself The time is now the time is now to express and for people to believe in themselves The time is now for it to be okay to be great People in this world shun people for being great for being a bright color for standing out but the time is now to be okay to be the greatest you You can talk yourself out of your destiny


Negative words can keep you from becoming who you were created to be don't fall into that trap Quit calling in defeat quit talking about how it's not going to happen You should wipe down your Write down what you want to see happen in life Any areas that you're struggling in where you need to improve write it down like it's already done and then every day declare that decree


convolution is a measure of overlap between two functions as one slides over the other mathematically it's a sum of products the standard convolution operation is slow to perform however we can speed this up with an alternative method that is the topic of this video depth wise separable convolution let's first very quickly go over the basics of convolution on an input volume consider

Depthwise Separable Convolution - A FASTER CONVOLUTION!


an input volume f.o shape d f cross d f cross m where DF is the width and height of the input volume and M is the number of input channels if a color image was an input then M would be equal to 3/4 the RG and B channels we apply convolution on a kernel K of shape DK cross DK cross M this will give us an output of shape D G cross DG cross 1 if we apply n such kernels on the input


then we get an output volume G of shape DG cross DG cross n the convolution operation takes the sum of products of the input and the kernel to return a scalar this operation is continued by sliding the kernel over the input I've explained this concept in detail on my video on convolution neural networks check that out for a clear understanding I'm more concerned now with the cost of


this convolution operation so let's take a look at that we can measure the computation required for convolution by taking a look at the number of multiplications required so why is that it's because multiplication is an expensive operation relative to addition so let's determine the number of multiplications for one convolution operation the number of multiplications


is the number of elements in that kernel so that would be D K times D K times M multiplications but we slide this kernel over the input we perform DG convolutions along the width and DG convolutions along the height and hence D G cross DG convolutions over all so the number of multiplications in the convolution of one kernel over the entire input f is DG square times D K


square times M now this is for just one kernel but if we have n such kernels which makes the absolute total number of multiplications become n times D G square times D K square times M multiplications let's now take a look at depth wise separable convolutions in standard convolution the application of filters across all input channels and the combination of these values are done


in a single step def y separable convolution on the other hand breaks us down into two parts the first is depth wise convolution that is it performs the filtering stage and then point wise convolution which performs the combining stage let's get into some details here depth wise convolution applies convolution to a single input channel at a time this is different from the


standard convolution that applies convolution to all channels let us take the same input volume F to understand this process F has a shape D F cross D F cross M where D F is the width and height of the input volume and M is the number of input channels like I mentioned before for depth wise convolution we use filters or kernels K of shape DK cross DK cross one here


DK is the width and height of the square kernel and it has a depth of 1 because this convolution is only applied to a channel unlike standard convolution which is applied throughout the entire day and since we apply one kernel to a single input channel we require M such DK cross DK cross one kernels over the entire input volume F for each of these M convolutions we end up with an output


DG cross DG cross one in shape now stacking these outputs together we have an output volume of G which is of shape DG cross DG cross M this is the end of the first phase that is the end of depth wise convolution now this is succeeded by point wise convolution point wise convolution involves performing the linear combination of each of these layers here the input is the volume of


shape DG cross DG cross M the filter K PC has a shape one cross one cross M this is basically a 1 Cross 1 convolution operation over all M layers the output will thus have the same input width and height as the input D G cross DG for each filter assuming that we want to use some n such filters the output volume becomes D G cross DG cross n so that's great we got this down now let's


take a look at the complexity of this convolution we can split this into two parts as we have two phases first we compute the number of multiplications in depth wise convolution so here the kernels have a shape DK cross D K cross 1 so the number of multiplications on one convolution operation is all DK times DK DK square when applied over the entire input channel this convolution is performed DG


x DG number of times so the number of multiplications for the kernel over the input channel becomes DG square times DK square now such multiplications are applied over all em input channels for each channel we have a different kernel and hence the total number of multiplications in the first phase that is depth wise convolution is M times D G square times D K square next we compute


the number of multiplications in the second phase that is point wise convolution here the kernels have a shape one cross one cross M where m is the depth of the input volume and hence the number of multiplications for one instance of convolution is M this is applied to the entire output of the first phase which has a width and height of D G so the total number of


multiplications for this kernel is d G times D G times M so for some n kernels will have n times D G times D G times M such multiplications and thus the total number of multiplications is the sum of multiplications in the depth wise convolution stage plus the number of multiplications in the point-wise convolution stage we can take M times D G squared common now we compare the


standard convolution with depth wise convolution we get the ratio as the sum of reciprocal of the depth of output volume that is n and the reciprocal of the squared dimensions of the kernel DK to put this into perspective of how effective depth wise convolution is let us take an example so consider the output feature volume n of 1024 and a kernel of size 3 that's DK is equal to 3


plugging these values into the relation we get zero point 1 1 2 in other words standard convolution has 9 times more the number of multiplications as that of depth Y separable convolution this is a lot of computing power we can also quickly compare the number of parameters in both convolutions in standard convolution each kernel has k times D K times M learn about


parameters since there are n such kernels there are n times M times D K squared parameters in depth by separable convolutions will split this once again into two parts in the depth wise convolution phase we use M kernels of shape DK cross DK in point wise convolution we use n kernels of shape 1 Cross 1 cross M so the total is M times DK square plus M times n or we can just


take M common taking the ratio we get the same ratio as we did for computational power required so we understood exactly what depth wise convolution is and also its computation power with respect to the traditional standard convolution but where exactly has this been used well there are some very interesting papers here the first is on multi model neural networks these


are networks designed to solve multiple problems using a single network a multi model network has four parts the first is modality Nets to convert different input types to a universal internal representation then we have an encoder to process inputs we have a mixer to encode inputs with previous outputs and we have a decoder to generate outputs a fundamental component of each of these


parts is depth wise separable convolution it works effectively in such large networks next up we have exception a convolution neural network architecture based entirely on depth wise separable convolution layers it has shown the state-of-the-art performance on large datasets like Google's jft image data set it's a repository of 350 million images with 17,000 class labels


to put this into perspective the popular image net took 3 days to Train however to Train even a subset of this jft data set it took a month and it didn't even converge in fact it would have approximately taken about three months to converge how'd they let it run to its full length so that's useful this paper is pushing convolution neural networks to use depth Y separable


convolution as the de facto up third we have mobile Nets a neural network architecture that strives to minimize latency of smaller scale networks so that computer vision applications run well on mobile devices mobile nets used F Y separable convolutions in its 28 layer architecture this paper compares the performance of mobile nets with fully connected layers versus depth wise


separable convolution layers it turns out the accuracy on image net only drops a 1% while using significantly less number of parameters from twenty nine point three million the number of parameters it's down to just 4.2 million we can see the mulch as the number of multiplications and additions which is a direct measure of computation has also significantly decreased for depth by


separable convolution mobile Nets so here are some things to remember in this video depth Y separable convolution decreases the computation and number of parameters when compared to standard convolution second is that depth Y separable convolution is a combination of depth wise convolution followed by a point wise convolution depth wise convolution is the filtering step and


point wise convolution can be thought of as the combination step finally they have been successfully implemented in neural network architectures like multi model networks exception and mobile nets and that's all I have for you now thank you all for stopping by today if you liked the video hit that like button if you want to stick around hit that subscribe button


the following is a conversation with Ian good fellow he's the author of the popular textbook on deep learning simply titled deep learning he coined the term of generative adversarial networks otherwise known as Ganz and with his 2014 paper is responsible for launching the incredible growth of research and innovation in this subfield of deep learning he got his BS and MS at

Ian Goodfellow: Generative Adversarial Networks (GANs) | Lex Fridman Podcast #19


Stanford his PhD at University of Montreal with yoshua bengio and Erin Kerrville he held several research positions including an open AI Google brain and now at Apple as the director of machine learning this recording happened while Ian was still a Google brain but we don't talk about anything specific to Google or any other organization this conversation is part


of the artificial intelligence podcast if you enjoy it subscribe on YouTube iTunes or simply connect with me on Twitter at lex friedman spelled fri d and now here's my conversation with Ian good fellow you open your popular deep learning book with a Russian doll type diagram that shows deep learning is a subset of representation learning which in turn is a subset of machine learning


and finally a subset of AI so this kind of implies that there may be limits to deep learning in the context of AI so what do you think is the current limits of deep learning and are those limits something that we can overcome with time yeah I think one of the biggest limitations of deep learning is that right now it requires really a lot of data especially labeled data there's


some unsupervised and semi-supervised learning algorithms that can reduce the amount of labeled data you need but they still require a lot of unlabeled data reinforcement learning algorithms they don't need labels but they need really a lot of experiences as human beings we don't learn to play pong by failing at pong two million times so just getting the generalization ability better is one


of the most important bottlenecks and the capability of the technology today and then I guess I'd also say deep learning is like a of a bigger system so far nobody is really proposing to have only what you'd call deep learning as the entire ingredient of intelligence you use deep learning as sub modules of other systems like alphago has a deep learning model


that estimates the value function most reinforcement learning algorithms have a deep learning module that estimates which action to take next but you might have other components here basically as building a function estimator do you think it's possible you said nobody is kind of in thinking about this so far but do you think neural networks could be made to reason in the way symbolic


systems did in the 80s and 90s to do more create more like programs as opposed to functions yeah I think we already see that a little bit I already kind of think of neural nets as a kind of program I think of deep learning as basically learning programs that have more than one step so if you draw a flowchart or or if you draw a tensor flow graph describing your machine


learning model I think of the depth of that graph is describing the number of steps that run in sequence and then the width of that graph is the number of steps that run in parallel now it's been long enough that we've had deep learning working that it's a little bit silly to even discuss shallow learning anymore but back when I first got involved in AI when we used machine learning we were


usually learning things like support vector machines you could have a lot of input features to the model and you could multiply each feature by a different weight but all those multiplications were done in parallel to each other there wasn't a lot done in series I think what we got with deep learning was really the ability to have steps of a program that run in sequence


and I think that we've actually started to see that what's important with deep learning is more the fact that we have a multi-step program rather than the fact that we've learned a representation if you look at things like res nuts for example they take one particular kind of representation and they update it several times back when deep learning first really took off in the academic


world in 2006 when Geoff Hinton showed that you could train deep belief networks everybody who was under ested in the idea thought of it as each layer learns a different level of abstraction but the first layer trained on images learn something like edges and the second layer learns corners and eventually you get these kind of grandmother's cell units that recognize


specific objects today I think most people think of it more as a computer program where as you add more layers you can do more updates before you output your final number but I don't think anybody believes the layer 150 of the resin it is a grand grandmother cell and you know layer 100 is contours or something like that okay so you think you're not thinking of it as a singular


representation that keeps building you think of it as a program sort of almost like a state the representation is a state of understanding and yeah I think of it as a program that makes several updates and arrives it better and better understandings but it's not replacing the representation at each step its refining it and in some sense that's a little bit like reasoning it's not


reasoning in the form of deduction but it's reasoning in the form of taking a thought and refining it and refining it carefully until it's good enough to use do you think and I hope you don't mind we'll jump philosophical every once in a while do you think of you know a cognition human cognition or even consciousness as simply a result of this kind of cincuenta sequential


representation learning do you think that can emerge cognition yes I think so consciousness it's really hard to even define what we mean by that I guess there's consciousness is often defined as things like having self-awareness and that's relatively easy to turn into something actionable for a computer scientists the reason about people also defined consciousness in terms of having


qualitative states of experience like qualia and there's all these philosophical problems like could you imagine jambe who does all the same information processing as a human but doesn't really have the qualitative experiences that we have that sort of thing I have no idea how to formalize or turn it into a scientific question I don't know how you could run in


experiment to tell whether a person is a zombie or not and similarly I don't know how you could run an experiment to tell whether an advanced AI system had become conscious in the sense of qualia or not but in the more practical sense like almost like self attention you think consciousness and cognition can in an impressive way emerge from current types of architectures though


yes yeah or or if if you think of consciousness in terms of self-awareness and just making plans based on the fact that the agent itself exists in the world reinforcement learning algorithms are already more or less forced to model the agents effect on the environment so that that more limited version of consciousness is already something that we get limited versions of with


reinforcement learning algorithms if they're trained well but you say limited so the the big question really is how you jump from limited to human level yeah right and whether it's possible you know the even just building common-sense reasoning seems to be exceptionally difficult so K if we scale things up forget much better on supervised learning if we get better at labeling


forget bigger datasets and the more compute do you think we'll start to see really impressive things that go from limited to you know something echoes of human level cognition I think so yeah I'm optimistic about what can happen just with more computation and more data I do think it'll be important to get the right kind of data today most of the machine learning systems we train our


mostly trained on one type of data for each model but the human brain we get all of our different senses and we have many different experiences like you know riding a bike driving a car talking to people reading I think when you get that kind of integrated data set working with a machine learning model that can actually close the loop and interact we may find that algorithms not so


different from what we have today learn really interesting things when you scale them up a lot and a large amount of multimodal data so multimodal is really interesting but within like you're working adversarial examples so selecting within modal within up one mode of data selecting better at what are the difficult cases from which are most useful to learn from


oh yeah like could we could you get a whole lot of mileage out of designing a model that's resistant to adverse fare examples or something like that right yeah question but my thinking on that has evolved a lot over the last few years one nice thing when I first started to really invest in studying adversarial examples I was thinking of it mostly as that versus aryl examples


reveal a big problem with machine learning and we would like to close the gap between how machine learning models respond to adversarial examples and how humans respond after studying the problem more I still think that adversarial examples are important I think of them now more of as a security liability then as an issue that necessarily shows there something


uniquely wrong with machine learning as opposed to humans also do you see them as a tool to improve the performance of the system not not on the security side but literally just accuracy I do see them as a kind of tool on that side but maybe not quite as much as I used to think we've started to find that there's a trade-off between accuracy on adversarial examples and accuracy on


clean examples back in 2014 when I did the first adversary trained classifier that showed resistance to some kinds of adversarial examples it also got better at the clean data on M NIST and that's something we've replicated several times an M NIST that when we train against weak adversarial examples Emnes classifiers get more accurate so far that hasn't really held up on other data


sets and hasn't held up when we train against stronger adversaries it seems like when you confront a really strong adversary you tend to have to give something up interesting this is such a compelling idea because it feels it feels like that's how us humans learn yeah the difficult cases we we try to think of what would we screw up and then we make sure we fix that yeah


it's also in a lot of branches of engineering you do a worst case analysis and make sure that your system will work in the worst case and then that guarantees that it'll work in all of the messy average cases that happen when you go out into a really randomized world you know with driving with autonomous vehicles there seems to be a desire to just look for think I'd viscerally tried


to figure out how to mess up the system and if you can be robust to all those difficult cases then you can it's a hand waving empirical way to show that your system is yeah yes today most adverse early example research isn't really focused on a particular use case but there are a lot of different use cases where you'd like to make sure that the adversary can't


interfere with the operation of your system like in finance if you have an algorithm making trades for you people go to a lot of an effort to obfuscate their algorithm that's both to protect their IP because you don't want to research and develop a profitable trading algorithm then have somebody else capture the gains but it's at least partly because you don't want people to


make adversarial examples that fool you our algorithm into making bad trades or I guess one area that's been popular in the academic literature is speech recognition if you use speech recognition to hear an audio waveform and then in turn that into a command that a phone executes for you you don't want and a malicious adversary to be able to produce audio that gets


interpreted as malicious commands especially if a human in the room doesn't realize that something like that is happening in speech recognition has there been much success in in being able to create adversarial examples that fool the system yeah actually I guess the first work that I'm aware of is a paper called hidden voice commands that came out in 2016 I believe and they were able


to show that they could make sounds that are not understandable by a human but are recognized as the target phrase that the attacker wants the phone to recognize it as since then things have gotten a little bit better on the attacker side when worse on the defender side it's become possible to make sounds that sound like normal speech but are actually interpreted as a different


sentence than the human here's the level of perceptibility of the adversarial perturbation is still kind of high the when you listen to the recording it sounds like there's some noise in the background just like rustling sounds but those rustling sounds are actually the adversarial perturbation that makes the phone hear a completely different sentence yeah that's so fascinating


Peter Norvig mention that you're writing the deep learning chapter for the fourth edition of the artificial intelligence the modern approach book so how do you even begin summarizing the field of deep learning in a chapter well in my case I waited like a year before I actually read anything is it even having written a full length textbook before it's still pretty


intimidating to try to start writing just one chapter that covers everything one thing that helped me make that plan was actually the experience of having ridden the full book before and then watching how the field changed after the book came out I realized there's a lot of topics that were maybe extraneous in the first book and just seeing what stood the test of a few years of being


published and what seems a little bit less important to have included now helped me pare down the topics I wanted to cover for the book it's also really nice now that the field is kind of stabilized to the point where some core ideas from the 1980s are still used today when I first started studying machine learning almost everything from the 1980s had been rejected and now some


of it has come back so that stuff that's really stood the test of time is what I focused on putting into the book there's also I guess two different philosophies about how you might write a book one philosophy is you try to write a reference that covers everything and the other philosophy is you try to provide a high level summary that gives people the language to understand a field and tells


them what the most important concepts are the first deep learning book that I wrote with Yahshua and Aaron was somewhere between the the two philosophies that it's trying to be both a reference and an introductory guide writing this chapter for Russell and Norvig book I was able to focus more on just a concise introduction of the key concepts and the language you need to


read about them more and a lot of cases actually just wrote paragraphs that said here's a rapidly evolving area that you should pay attention to it's it's pointless to try to tell you what the latest and best version of a you know learn to learn model is right you know I can I can point you to a paper that's recent right now but there isn't a whole lot of a reason to delve into exactly


what's going on with the latest learning to learn approach or the latest module produced by learning to learn algorithm you should know that learning to learn is a thing and that it may very well be the source of the latest and greatest convolutional net or recurrent net module that you would want to use in your latest project but there isn't a lot of point in trying to summarize


exactly which architecture in which learning approach got to which level of performance so you maybe focus more on the basics of the methodology so from back propagation to feed-forward to recur in your networks convolutional that kind of thing yeah yeah so if I were to ask you I remember I took algorithms and data structures algorithm there of course remember the professor asked what is an


algorithm and yelled at everybody in a good way that nobody was answering it correctly everybody knew what the alkyl it was graduate course everybody knew what an algorithm was but they weren't able to answer it well let me ask you in that same spirit what is deep learning I would say deep learning is any kind of machine learning that involves learning parameters of more than one consecutive


step so that I mean shallow learning is things where you learn a lot of operations that happen in parallel you might have a system that makes multiple steps like you might have had designed feature extractors but really only one step is learned deep learning is anything where you have multiple operations in sequence and that includes the things that are really popular today


like convolutional networks and recurrent networks but it also includes some of the things that have died out like Bolton machines where we weren't using back propagation today I hear a lot of people define deep learning as gradient descent applied to these differentiable functions and I think that's a legitimate usage of the term it's just different from the way that I


use the term myself so what's an example of deep learning that is not gradient descent on differentiable functions in your I mean not specifically perhaps but more even looking into the future what's your thought about that space of approaches yeah so I tend to think of machine learning algorithms as decomposed into really three different pieces there's the model which can be


something like a neural nut or a Bolton machine or a recurrent model and I basically just described how do you take data and how do you take parameters and you know what function do you use to make a prediction given the data and the parameters another piece of the learning algorithm is the optimization algorithm or not every algorithm can be really described in


terms of optimization but what's the algorithm for updating the parameters or updating whatever the state of the network is and then the the last part is the the data set like how do you actually represent the world as it comes into your machine learning system so I think of deep learning as telling us something about what does the model look like and basically to qualify as deep I


say that it just has to have multiple layers that can be multiple steps in a feed-forward differentiable computation that can be multiple layers in a graphical model there's a lot of ways that you could satisfy me that something has multiple steps that are each parameterised separately I think of gradient descent as being all about that other piece the how do you


actually update the parameters piece so you can imagine having a deep model like a convolutional net and training it with something like evolution or a genetic algorithm and I would say that still qualifies as deep learning and then in terms of models that aren't necessarily differentiable I guess Boltzmann machines are probably the main example of something where you


can't really take a derivative and use that for the learning process but you you can still argue that the model has many steps of processing that it applies when you run inference in the model so that's the steps of processing that's key so geoff hinton suggests that we need to throw away back prop back propagation and start all over what do you think about that what could an


alternative direction of training nil networks look like I don't know that back propagation is going to go away entirely most of this time when we decide that a machine learning algorithm isn't on the critical path to research for improving AI the algorithm doesn't die it just becomes used for some specialized set of things a lot of algorithms like logistic regression don't seem that exciting to


AI researchers who are working on things like speech recognition or autonomous cars today but there's still a lot of use for logistic regression and things like analyzing really noisy data and medicine and finance or making really rapid predictions in really time-limited contexts so I think I think back propagation and gradient descent are around to stay but they may not end up


being everything that we need to get to real human level or superhuman AI are you optimistic about us discovering you know back propagation has been around for a few decades so I optimistic bus about us as a community being able to discover something better yeah I am I think I think we likely will find something that works better you could imagine things like having stacks of models where some


of the lower level models predict parameters of the higher level models and so at the top level you're not learning in terms of literally calculating gradients but just predicting how different values will perform you can kind of see that already in some areas like Bayesian optimization where you have a Gaussian process that predicts how well different parameter


values will perform we already used those kinds of algorithms for things like hyper parameter optimization and in general we know a lot of things other than back prep that work really well for specific problems the main thing we haven't found is a way of taking one of these other non back based algorithms and having it really advanced the state-of-the-art on an AI level problem right but I


wouldn't be surprised if eventually we find that some of these algorithms that even the ones that already exists not even necessarily a new one we might find some way of customizing one of these algorithms to do something really interesting at the level of cognition or or the the level of I think one system that we really don't have working quite right yet is like short-term memory we


have things like LST M's they're called long short-term memory they still don't do quite what a human does with short-term memory like gradient descent to learn a specific fact has to do multiple steps on that fact like if I I tell you the meeting today is at 3 p.m. I don't need to say over and over again it's at 3 p.m. it's not 3 p.m. it's at 3 p.m. it's a 3 p.m. right for you to do a gradient


step on each one you just hear it once and you remember it there's been some work on things like self attention and attention like mechanisms like the neural Turing machine that can write to memory cells and update themselves with facts like that right away but I don't think we've really nailed it yet and that's one area where I'd imagine that new optimization algorithms are


different ways of applying existing optimization algorithms could give us a way of just lightning-fast updating the state of a machine learning system to contain a specific fact like that without needing to have it presented over and over and over again so some of the success of symbolic systems in the 80s is they were able to assemble these kinds of facts better but dude there's a


lot of expert input required and it's very limited in that sense do you ever look back to that as something that will have to return to eventually sort of dust off the book from the shelf and think about how we build knowledge representation knowledge place well we have to use graph searches searches right and like first-order logic and entailment and things like that a thing


yeah exactly in my particular line of work which has mostly been machine learning security and and also generative modeling I haven't usually found myself moving in that direction for generative models I could see a little bit of it could be useful if you had something like a differentiable knowledge base or some other kind of knowledge base where it's possible for some of our fuzzier machine


learning algorithms to interact with the knowledge base immanuel Network is kind of like that it's a differentiable knowledge base of sorts yeah but if if we had a really easy way of giving feedback to machine learning models that would clearly helped a lot with with generative models and so you could imagine one way of getting there would be get a lot better at natural


language processing but another way of getting there would be take some kind of knowledge base and figure out a way for it to actually interact with a neural network being able to have a chat within y'all network yes so like one thing in generative models we see a lot today is you'll get things like faces that are not symmetrical like like people that have two eyes that are different colors


and I mean there are people with eyes that are different colors in real life but not nearly as many of them as you tend to see in the machine learning generated data so if if you had either a knowledge base that could contain the fact people's faces are generally approximately symmetric and eye color is especially likely to be the same on both sides being able to just inject that


hint into the machine learning model without it having to discover that itself after studying a lot of data it would be a really useful feature I could see a lot of ways of getting there without bringing back some of the 1980s technology but I also see some ways that you could imagine extending the 1980s technology to play nice with neural nets and have it help get there


awesome so you talked about the story of you coming up with idea of Gans at a bar with some friends you were arguing that this you know Gans would work Jenner of adversarial networks and the others didn't think so then he went home at midnight coated up and it worked so if I was a friend of yours at the bar I would also have doubts it's a really nice idea but I'm very skeptical that it would


work what was the basis of their skepticism what was the basis of your intuition why he should work I don't want to be someone who goes around promoting alcohol for the science in this case I do actually think that drinking helped a little bit mm-hmm when your inhibitions are lowered you're more willing to try out things that you wouldn't try out otherwise so I I have


noticed it in general that I'm less prone to shooting down some of my own ideas when I'm when I have had a little bit to drink I think if I had had that idea at lunch time yeah I probably would have thought it it's hard enough I mean one neural net you can't train a second neuron that in the inner loop of the outer neural net that was basically my friends action was that trying to train two


neural nets at the same time would be too hard so it was more about the training process unless so my skepticism would be you know I'm sure you could train it but the thing would converge to would not be able to generate anything reasonable and any kind of reasonable realism yeah so so part of what all of us were thinking about when we had this conversation was deep Bolton machines


which a lot of us in the lab including me were a big fan of deep bolts and machines at the time they involved two separate processes running at the same time one of them is called the positive phase where you load data into the model and tell the model to make the data more likely the owners called the negative phase where you draw samples from the model and tell the model to make those


samples less likely in a deep Bolton machine it's not trivial to generate a sample you have to actually run an iterative process that gets better and better samples coming closer and closer to the distribution the model represents so during the training process you're always running these two systems at the same time one that's updating the parameters of the model and another one


that's trying to generate samples from the model and they worked really well on things like Amnesty a lot of us in the lab including me had tried to get the Boltzmann machines to scale past em inist to things like generating color photos and we just couldn't get the two processes to stay synchronized so when I had the idea for Gans a lot of people thought that the discriminator would


have more or less the same problem as the negative phase in the Boltzmann machine that trying to train the discriminator in the inner loop you just couldn't get it to keep up with the generator and the outer loop and that would prevent it from converging to anything useful yeah I share that intuition yeah what turns out to not be the case a lot of the time with machine


learning algorithms it's really hard to predict ahead of time how well they'll actually perform you have to just run the experiment and see what happens and I would say I still today don't have like one factor I can put my finger on it say this is why ganz worked for photo generation and deep Boltzmann machines don't there are a lot of theory papers showing that under some theoretical settings the


the gun algorithm does actually converge but those settings are restricted enough that they don't necessarily explain the whole picture in terms of all the results that we see in practice so taking a step back can you in the same way as we talked about deep learning can you tell me what generative adversarial networks are yeah so generative adversarial networks are a particular


kind of generative model a generative model is a machine learning model that can train on some set of data like so you have a collection of photos of cats and you want to generate more photos of cats or you want to estimate a probability distribution over cats so you can ask how likely it is that some new image is a photo of a cat ganzar one way of doing this


some generative models are good at creating new data other generative models are good at estimating that density function and telling you how likely particular pieces of data are to come from the same distribution as a training data gans are more focused on generating samples rather than estimating the density function there are some kinds of games like flow gun


that can do both but mostly guns are about generating samples of generating new photos of cats that look realistic and they do that completely from scratch it's analogous to human imagination when again creates a new image of a cat it's using a neural network to produce a cat that has not existed before it isn't doing something like compositing photos together you're not you're not literally


taking the eye off of one cat on the ear off of another cat it's it's more of this digestive process where the the neural net trains on a lot of data and comes up with some representation of the probability distribution and generates entirely new cats there are a lot of different ways of building a generative model what's specific against is that we have a two-player game in the game


theoretic sense and as the players in this game compete one of them becomes able to generate realistic data the first player is called the generator it produces output data such as just images for example and at the start of the learning process it'll just produce completely random images the other player is called the discriminator the discriminator takes images as input and guesses whether