all kinds of stuff stupid chat BOTS you know that are entertaining but we can't have things that you know we the technology we have is not enough to get machines that have common sense to get to build things like intelligent personal assistants that really help us it can help us in the daily lives answer any question we have and you know kind of be a bit more like like human

Self-Supervised Learning


assistance we can have really smart chat BOTS we can have household robots that you know take care of all the chores in their house we don't really have a jar in dexterous robots they are a giant dexterous and very kind of specific situations but it's sort of very brittle and we can't have artificial intelligence so in general we can't have artificial general intelligence because

Self-Supervised Learning


that concept does not exist there is no such thing as general intelligence and I hate this term AVI there's a lot of people who claim that you know they are going to get to AGI by scaling up reinforcements running just having more computation this is completely false okay those people are after investment so they're ready to either you know self be so deluded or or kind of stretch the

Self-Supervised Learning


truth a little bit but in my opinion we're not gonna get there with the current type of running that we are that we're using and so why is there no such thing as artificial general intelligence and that's because there is no general intelligence human intelligence is incredibly specialized I'm sorry to say that okay that applies to everyone in this room but our intelligence is super

Self-Supervised Learning


specialized you know we're built by hit by evolution to kind of survive in our environment and we have this sort of impression that our intelligence in general but we just suck at a lot of tasks okay and in fact a lot of the tasks that computers can do quite well we totally suck at it so there was this idea that you know before alphago if I go 0 etc that humans were

Self-Supervised Learning


the best go player in the world were very very close to the ideal player okay good god alright that you could you could get just a few stones of handicap with idea player and basically beat the idea player two or three stones and you care for something like this turns that no turns out the best human players are horrible you know current machines are much much better than they are like by a

Self-Supervised Learning


huge margin so we just suck at it we're really bad which means you know that's not part of the stuff that evolution kind of built into our our brain to be able to do well now the thing is the reason why people were thinking that you know they were very close to the ideal player was because they could not imagine you know smarter considerably smaller entities and so we cannot

Self-Supervised Learning


imagine all the stuff that we're not able to do and therefore we think of ourselves as having general intelligence it's just that or imagination for what you know what functions we need to be able to do is very limited let me give you another more specific example anymore I wouldn't say mathematical but it's you more quantitative your your optical nerve has 1 million fibers so imagine we

Self-Supervised Learning


just take the 1 million fibers coming out of your optical nerve that goes to your brain and imagine that they're just binary so what you see is just a binary image ok 1 million bits how many so a particular recognition function if you want ok recognizing your grandmother or whatever is a boolean function 1 million bits in the input and 1 bit in the output and the question is how many

Self-Supervised Learning


search functions are there anyone has any idea how many search function how many boolean functions of 1 million bits are there any suggestion any idea to to the 1 million yet you are off by a huge factor but it's a good start 25 yes 2 to the 2 to the 1 million that's a correct answer ok so you have 2 to the 1 million input configuration of 1 million bits right and for each of

Self-Supervised Learning


those 2 to the 1 million configuration you have one output bit that's the truth table of a particular boolean function ok so the number of configurations of 2 to the 1 million bits is 2 to the 2 to the 1 million it's an adorably large number I mean it's just a ridiculously large number now among all of those functions how many do you think what proportion do you think your brain can

Self-Supervised Learning


actually compute your visual cortex visual cortex has you know order between 10 and 100 billion neurons order 10 to the 14 synapses okay so it has 10 to the 14 synapses let's say to be generous each synapse scanning can store 10 bits ok so there's 10 to the 15 bits in your entire visual cortex that's your that's what determines the function of you know of your your visual cortex that means

Self-Supervised Learning


the number of functions your visual cortex can possibly implement is 2 to the 10 to the 15 that's a lot less than 2 to the 2 to the 1 million not a lot less it's just you know this is like no comparison right one so the number of functions that your visual cortex can implement compared to all possible functions it's just this tiny tiny tiny sliver we're super specialized in

Self-Supervised Learning


particular if I play a trick on you I cut your article nerve I'm gonna do it ok and I put a device between your your retina and your brain that permutes all the pixels in your head in your optical nerve with a random permutation but a fixed one ok so now there is no spatial consistency in the signal you get to your visual cortex I don't think you can envision because your cortex has local

Self-Supervised Learning


connections and those local connections are there to exploit local correlation and now you break the circle correlation but doing this permutation you can you can see anyone you might see at very low resolution because the higher layers have big context but what is it true what is true yes okay so it is retin-a topic so the connection between the the optical nerve and the

Self-Supervised Learning


visual cortex is within a topic which means the topology is preserved the connections are largely local there are long range collections but this is only a small number of them and so you don't you don't have a huge amount of you know communication bandwidth for the long range you you have you know big bundles of connections from sort of low layers to high riders if you want from V 1 to V

Self-Supervised Learning


2 and V 2 to V 4 and once you get to the higher layers the the spatial distribution is not represented anymore it's like a cornet where you are pulling and so in the high high layers you don't need that organization but by the time you get there the spatial resolution is lost so we can do the experiment actually it would be fun ok so next question is how do humans and

Self-Supervised Learning


animals learn so I don't know how many of you were were here yesterday at the inauguration probably not many but there is this idea that humans learn in a very different way from either reinforcement or supervised running and I'll call this later so supervised running but this is just a hypothesis but if you but you know babies learn concepts they learn sort of basic facts about basic

Self-Supervised Learning


knowledge about the world basically just by observation in the first few weeks and months of life and Emanuel depuis who is a colleague of Jean and me Vaness and on Facebook put together this chart that shows at what age babies learn different concepts so things like being able to make the difference between animate and inanimate object that pops up around three months and the

Self-Supervised Learning


notion of object permanence the fact that an object that is hidden behind another one is still there still exists the notions of solidity rigidity stability and then intuitive physics like gravity inertia are things like this pop up around eight months so if you show a six-month-old baby the scenario on the top left where you put a little car on a on a platform and you

Self-Supervised Learning


push up you push the car off the platform and it doesn't fall it's hidden in the back put the baby you can see that it's a trick six months they they're not surprised to just you know that's how the world works it was one more thing I need to learn after nine months they've learned that objects are not supposed to float in the air that you're supposed to fall and they go like

Self-Supervised Learning


this okay and you can measure how long they stare at it and with how much attention and so that's how you know that a concept has been has been known on that we know if a concept is is violated by a particular scene that you show the baby the baby is going to be really surprised and you can measure the degree of surprise if you want so how is it that babies learned is just basically

Self-Supervised Learning


by observation you know young babies you know before a few months old or completely helpless I just observed and don't really have any way of affecting the physical world around them so how does that happen so it's a different type of learning that either refers mental supervised ready and it's not just babies it's you know almost animals learn learn this kind of stuff this is a

Self-Supervised Learning


baby or wrong.we Tong is being shown a magic trick whether it's a an object in the cup and the object is removed but he doesn't see that and now the cup is empty and he's running on the floor laughing so obviously you know his model of the world includes object permanence and objects are not supposed to disappear like this and you know and when we see something that surprises us

Self-Supervised Learning


we are you know we laugh or we get scared because here is something we didn't predict and it can kill us so it's all kinds of concepts like this the reason for this animation here at the top is is that these very basic concepts like the fact that the world is filled emotional that perhaps we can learn but there's training of cells to predict very simple things so if I if I train

Self-Supervised Learning


myself at train my brain or if I train a running machine to predict what the world is going to look like when I'm when I move my my head or the camera a few centimeters to the left the view of the world changes depending on you know objects move with parallax depending on the depth the distance to my eyes and so if I train myself to predict what the world's going to look

Self-Supervised Learning


like when I move the camera perhaps I can automatically infer that every object in the world has a depth because that's the best explanation for the simplest explanation for how things change okay so the notion of depth the fact that the world is feel emotional might just simply emerge from training ourselves to predict what the world looks like when we move our head once

Self-Supervised Learning


you have that you have occlusion edges you know objects that are nearby don't move the same way that objects that are far away and so you see them as as objects okay there's a bunch of you know so weekly supervised vision systems that exploit this this kind of this kind of property once you have objects you know you have objects they can they can move independently of others your background

Self-Supervised Learning


objects you have you know the notion of obstacle you know things like that localization so you could think of you know concepts like this being sort of built hierarchically by just training yourself to predict and then representing you know coming up with good representations of that allow you to do a good job at predicting okay so this is not supervised it would be an

Self-Supervised Learning


unsupervised way form of running and that led some of us to so this is the joke from earlier shell force it's a play on some poem from the 1960s or 70s the variation will not be supervised so the future is in a new form of learning that will allow a machine to accumulate all those background knowledge about how the world works mostly by observation a little bit by interaction but mostly

Self-Supervised Learning


without supervision mostly without reinforcement so maybe that's the salvation so supervised learning what is it the the basic concepts the basic concept is I'll give you a piece of data let's say a piece of video for the sake of being concrete and I'm going to mask a piece of that video perhaps the second half of the video which is in blue at the top I'm going to

Self-Supervised Learning


train a machine to predict the future of the video from the past and the present okay but the general concepts of supervised learning is you have a piece of data you mask a piece of it and you ask the machine to predict the piece that is masked from the piece that is not mask if the piece that is masked is always the same so is the future for example you know you can use a some sort

Self-Supervised Learning


of prediction architecture for that but more often than not you don't actually know which piece is gonna be masked for example in this scene here right now you don't see my back but you can have you might have some good idea that you know what it looks like and you know maybe your brain sort of unconsciously tries you know it's predicting what I look like from the back

Self-Supervised Learning


and once I turn around you know your belief about this is updated it you can you can train yourself you can train your model you know same for all kinds of parts of the scene here which are currently occluded from your from your view so this principle of sort of learning to predict things that you will eventually see I think is a good one now this is so again you could train

Self-Supervised Learning


yourself to predict the past from the present to predict the top of the image from the bottom you know whatever doesn't doesn't matter exactly what it is the advantage of this is that this is something that you know Jeff Fenton has claimed for a long time is that the amount of information you're giving to the machine at every time step but every trial at every sample is enormous you're

Self-Supervised Learning


asking to predict every pixel you know in a bunch of frames in a video which is a lot of information much more than the label of a of an image for example which means you're putting a lot more constraints on the parameters of the machine which means you can train the machine to learn a lot of knowledge with a relatively small number of samples and furthermore those samples are free

Self-Supervised Learning


because you know we have more video data than then we can deal with so essentially if you think about sort of a hierarchy of the type of loading paradigms that we've been talking about here in South supervisors running there's a huge amount of feedback you're giving to the machine you know you're giving it a piece of video and then you were asking it you're telling it you know predict

Self-Supervised Learning


all those pixels it's an enormous amount of information this is a technical issue with it which I'll come back I'll come to in a minute supervise running you give a relatively small amount of feedback you tell the Machine this is class number three out of a thousand it's not a huge amount of information and I should say right now the the reason why neural nets work so well on

Self-Supervised Learning


image net is not just that it has 1 million training samples it's that it has 1,000 categories having a problem with lots of categories helps a lot to kind of construct good representations and then reinforcement learning is a very very weak feedback you're only telling the Machine once in a while you got it right or you get it wrong you're giving just a scalar value

Self-Supervised Learning


it's absolutely no way that a machine can learn anything complex without lots and lots of interactions using basic reinforcement running it's just no way you're just not giving it a lot of information yeah you know that's what you know a learning theory the only theory is called this sample complexity and it's just completely obvious that there is no way you can run complex

Self-Supervised Learning


stuff without tons and tons of interactions when you're giving just one scalar value once in a while as a feedback so the path to human intelligence you know my go through reinforcement running but it's not gonna be necessary it's not gonna be sufficient that's for sure that led me to this obnoxious analogy of intelligence as a cake and you know if so supervised running is the

Self-Supervised Learning


bulk of the cake machine learning is in the same embarrassing situation as physics in the sense that physicists have no idea what 95% of the mass in the universe is you know it's dark matter and dark energy they have no idea what it is we only know the 5% that is actually real matter and but the rest we don't know what it is so here's the same thing we can do we can

Self-Supervised Learning


make the chair you can make the icing on the cake but we can't actually bake the cake okay so I give you a preview earlier of what's missing and Wes missing is the ability to learn models predictive models of the world you know in the example of the car you if you want to train your system to drive a car it has to have some sort of politi model of what's gonna happen so as not to try

Self-Supervised Learning


stupid things like running into a tree or off a cliff so sorry years ago a few of my colleagues at Facebook ran this experiment which they you know they did so cooked up a bunch of simple physical situations where you stack a bunch of cubes and then you either run a game engine to can assimilate the cubes following or not following or you actually have real data where you take

Self-Supervised Learning


videos of stack of cubes and you train a video version of cornet basically to predict what's going to happen to the cubes so what you see here is what actually happens those are so the segmentation maps of the various cubes and it says what the continent is producing and the predictions are kind of blurry the reason why they vary is because there is no there's no way of

Self-Supervised Learning


exactly predicting what's going to happen there is a little bit of uncertainty about where other cubes actually are and everything and so what the system produces is a prediction which is sort of an average of multiple futures that can happen and that's a blurry prediction so how to deal with uncertainty is going to become the the main the main problem here right we're gonna make we're going

Self-Supervised Learning


to predict what's going to happen in the world by doing video prediction or we're going to take an image or a piece of text mask some parts of it and then ask a system to reconstruct and there is no way to make those predictions exactly and so what we had to have our systems that can deal with the uncertainty in the prediction they can represent the uncertainty of the prediction and it's

Self-Supervised Learning


for that reason to introduce the notion of energy based learning so you can think of it as kind of a weaker form of learning that people are used to which is sort of learning probabilistic models or learning the cities running you know distributions and the reason that we have to weaken it is because in high dimensional continuous spaces like images we don't have good ways of

Self-Supervised Learning


representing distributions that mean anything useful so let's say our entire world consists of two scalar variables y1 and y2 we're not going to do prediction here I mean we could consider that one variables observe the other one is not but we don't know in advance which one so let's say you observe y2 okay and this is this is our training set each each each point here is a

Self-Supervised Learning


training sample so this is obviously some structure in our world here you know all the values of y1 y2 seem to rely on some sort of surface line I mean your curve and if I give you a value of y2 you can predict that the value of y1 will be sort of around here or around there this is this is 30 okay I couldn't tell it was like a comma or not ok 30 minute that's perfect

Self-Supervised Learning


so right so there are multiple possible predictions so if you train a neural net or whatever primary trace function to make one prediction of y2 of y1 as a function of Y - it's not gonna work because you can only predict one output if you train the system willie square when it sees half of the sample on this side the other half on that side what is going to produce is the mean of the two

Self-Supervised Learning


okay that's the best way to minimize the square error that's not a good prediction for y1 for this value of y2 right that's those blurry predictions I was showing you about that's the brewery prediction here right in the middle so how do we kind of turn a prediction with sort of multiple possible outputs into kind of an architecture if you want and the proposal of energy based model so if

Self-Supervised Learning


you of course if you are probabilities you say well you know this is just a joint distribution I'm just going to learn the density of the Joint Distribution between those two variables and I'm done yeah you can do this in two dimensions you can't do this in 1 million dimensions when the those those things represent natural images for example so what I'm proposing is we're

Self-Supervised Learning


going to learn a energy function as its energy function so think of it as the negative log of a probability but it's not going to be normalized we're not gonna care about normalization okay so that in that way it's a little more general then then then probably stick approaches so give if if our data are those blue beads here an appropriate energy function that captures the

Self-Supervised Learning


dependency between the two variables is look something like this it takes low energies on the on the samples and higher energies outside okay and if we have a system like this if we have a function that has two inputs in this case when I put that gives us the basically the compatibility between the two inputs the two values that we give it we can use it to predict you know

Self-Supervised Learning


I'll give you a value of y2 and then buy gradient descent you can find or buy some search our method you can you can find the two values that produce a low energy on the output and they correspond to the two values of by one that are compatible way to okay so that's that's how inference works in those in those systems the system doesn't produce an output it only has

Self-Supervised Learning


inputs if I could strain the value of some of the inputs I can compute the value of the other inputs that will minimize the energy using some some scheme so now the second question is how do you train this box that produces the energy function and the training will do things like shaping the energy function so that it will take low energy on the on the blue beads and higher energies

Self-Supervised Learning


outside so if you have a primary Trice function say in the form of neural net that produces a scalar output it's very easy to show it a sample and then tune the parameter so the output goes down right so show it a sample one of the blue beads and then tweak the parameters on your net so that the output goes down so you get low energy four data points another second question is how do you

Self-Supervised Learning


make sure the energy is higher outside because if it's if the energy function is flat it just gives you low value for everything it doesn't play any interesting role so you have to make sure the energy is higher outside of the region of data and this is something that probably stick model give you know like maximum likelihood and normalize probabilistic model do automatically and

Self-Supervised Learning


the way you transform an energy based model into into a public one is it's really gives distribution I mean you have several ways but that's kind of a very natural when to do so take your energy function and take the exponential minus multiplied by some arbitrary positive constant and then normalize so you get a bunch of numbers between zero and one that's sum to one okay

Self-Supervised Learning


in the discrete case that's that's called soft max in the continuous case it's it's called a gives distribution in the general case where the energy function is complicated this normalizing turn normalization term is intractable you can't compute this integral so you can't actually turn the energy into into a probability and that's why I'm arguing for just manipulating the energy function now

Self-Supervised Learning


going to a density because you can't normalize it you can only normalize when the energy is kind of has a trivial form and it's not that interesting so right so that's the Department of Energy baserunning it's easy to make the energy low on the samples it's higher to make it it's harder to make sure it's higher outside it's possible to interpret classic running algorithms in terms of

Self-Supervised Learning


sort of energy type models you know classic and supervisor algorithms like PCA or k-means or gets a mixture model or square ICA things like this those are two examples here for PCA and k-means so this is in two dimensions the variable has is just a vector with two dimensions and the energy function for 4k means is for PCA I'm sorry it's just the square root construction error so you take a

Self-Supervised Learning


vector Y but you know project it on the principle subspace which is done by this matrix W and then multiply by W transpose and assuming things are appropriately normalized you get a reconstruction of the original point which is just in the original space the location of a projection of a point onto the onto the mat the linear subspace the principal subspace so this is the

Self-Supervised Learning


principal subspace here in dark and you take any point you project it here and the energy now the reconstruction error which is the distance between the original point on this projection on a principal subspace and so the the grayscale here represent that energy okay zero energy on the principal subspace and energy that goes quadratic quadratically as you move away obviously

Self-Supervised Learning


this is not a good representation of that data manifold this is the data okay so the data points are sampled from this spiral and PCA doesn't do a good job at this right k-means has a funny kind of energy function which is not directly doesn't have a sort of direct form but it's the it's the minimum of some more elementary energy function over some latent variable Z and we'll come back to

Self-Supervised Learning


models of this type so we have a energy function which is a reconstruction error so it's the squared distance between a data point Y and its reconstruction and its reconstruction is the product of a prototype matrix where the columns are bunch of prototypes multiplied by Z vector which is a latent variable and that Z vector is constrained to be a one hot vector so it's a vector with all

Self-Supervised Learning


zeros except it has a one at one location and to measure the reconstruction error of a particular point you figure out which prototype is closest to it and then the reconstruction error is the distance between the data point and the reconstruction and the the the closest prototype okay so the minimizing with respect to the Z vector will figure out which column of W which is which

Self-Supervised Learning


prototype is close as to why and that's the that's the one that produces the lowest reconstruction error and so now the energy function is defined as the minimum of Z where Z is a word of one of K code of this energy function now when you plot this energy will you train k-means on this data set where the data is sampled from this pink spiral and yet I think 2020 prototypes here you get a

Self-Supervised Learning


whole bunch of potential wells if you want so the quadratic bowls energy minima and the overall energy is the minimum of all of those twenty quadratic balls it looks really beautiful in two dimensions it doesn't scale very well within in high dimension because you know how do you populate a high dimensional space with photo types okay so those are merely two examples of

Self-Supervised Learning


classical and supervised Ling methods but here what I've done is list seven different classes or methods that ensure that the energy outside the region of of data is higher than on the of data so I mentioned the one at the top I build a machine so that the volume of low energy stuff is constant and that's the case for PCA for k-means gas mixture model square C etc another one

Self-Supervised Learning


is you make your energy function very flexible but you think of it as the log of some probability and you do maximum likelihood so automatically because your probability distribution needs to be normalized it's going to have the effect of pushing up the energy of stuff you don't observe but it's very difficult because you get the log of this normalization term the partition

Self-Supervised Learning


function which is generally intractable and so it's very hard to do this so that's called maximum likelihood and which when it's intractable you have to use approximation like variational approximations or or Monte Carlo methods or Markov chain Monte Carlo methods another technique which is quite popular is you push down on the energy of data points and then you push up on chosen

Self-Supervised Learning


locations outside and if you are familiar with Gans generated referral networks john chavis all networks our way of doing this so think of the object you're training it again as the discriminator not the generator the discriminator is the thing you're actually training and we think about the discriminator it is an energy function ok we move the exponential at the end of

Self-Supervised Learning


your audio discriminator it just produces a score think of this score as an energy that you're going to try to make large for bad samples and low for good samples okay good for you know low for examples you observe and bad for samples you don't observe and the question is how do you generate bad samples whose energy going to push up and the idea of gann is that you train a

Self-Supervised Learning


neural net which from a bunch of random numbers is going to produce a bad samples whose energy you're gonna push up that's your generator ok so you get and in fact a paper on this called energy based Ganso it's gonna be interpretation against in the context of energy based models and we talked about the other ones except except those two so this one denoising auto-encoder says

Self-Supervised Learning


I'm not going to train the system to actually compute an energy function it's physically but I'm gonna train a dynamical system to start from a point outside the region of data and then I'm going to train a neural net to map it back to the region of data okay so I take a training sample I corrupt it so now I take a noisy training sample and I train a neural net to map it back so my

Self-Supervised Learning


reconstruction error now is the distance between the noisy sample and the original one and so it's going to make the energy kind of be large outside the the region of data and I'll come back to this a little more explicitly but my favorite one is the last one and that last one says we're going to regularize some parameter inside the network so that the volume of stuff that is

Self-Supervised Learning


properly reconstructed is low okay so we're going to make the system pay for reconstructing too many things and how to do this and I'll come to this in a minute so if you are if you are probabilities and you go through this you do maximum likelihood you have you know you you your energy function as the normalized log of some probability distribution so you go through the gives

Self-Supervised Learning


distribution to turn it into a distribution into a normalized distribution and now you have a bunch of data points what you want to do is maximize the probability that your model gives to your data points so the product of the probability that your model gives to the data point or you want to minimize the negative log of the probability that your model gives to all

Self-Supervised Learning


the data points and that's here the objective function you see here so this is the negative log like if you want of a data point it's the negative log of the numerator also divided by data because it's easier minus the negative log of the denominator okay but there's two minuses so that gives us surprise so that's the objective function in each minimize if you are probably list and

Self-Supervised Learning


this will have the effect of pushing down on the energy of the data points and this will have the effect if minimize this you'll have to push up on the energy of every single point in your space okay because to make this small you have to make those energies high because it's negative exponential if you compute the gradient of that last function with respect to the parameters

Self-Supervised Learning


of your energy function you get the expression at the top here which tells you that a step of gradient is going to push down with a unit force on the data point and then the second term says I'm going to push up on every point in the space with a force that's proportional to the probability that my model gives to that point okay so points I have high probability gonna get pushed up really

Self-Supervised Learning


high points that I have low probability which means high energy I'm gonna push that not as not as hard the integral of the force is one and so when the only data point that has high probability which means low energy is the correct one then those two terms balance and the thing can converges if you want you can't actually compute this integral so very often you have to approximate it

Self-Supervised Learning


using Monte Carlo methods or variation of approximations and this is what a lot of people instead of probabilistic modeling do there's a lot of papers on this but you don't need to do all this ok so now let's talk about latent variable models so I talked about the k-means method where this is Z variable that you have to minimize over to get the energy of your of your system and

Self-Supervised Learning


it's kind of specific example of a kind of more general approach ignore the stuff on the on the right for now just look at the left side so you can either prefer the equations or the block diagram depending on whether you are a mathematician and computer scientist or an engineer but it's okay the block diagram now so you have data point Y that you want to reconstruct and

Self-Supervised Learning


your energy function is going to be the reconstruction error you're going to reconstruct this data point by running a latent variable through a decoder function think of it as on your own net in the simple sketch is just a simple matrix W code dictionary matrix and then you compute the square error so in this case the C cost function here is just a square error between the data points and

Self-Supervised Learning


the multiplication of a matrix by latent vector okay this is like a means except we don't constrain this vector to be one of K and there's a problem with this if you just use this without anything else if Z has the same dimension as Y your bigger or even slightly smaller there's always going to be a wire that's going to reconstruct ie it is always going to be a Z that is going to perfectly

Self-Supervised Learning


reconstruct any way you throw at it okay for a non degenerate version of the decoder nondegenerate matrix here is always going to be a Z that exactly we constructed the Y that's not good because that means every point in your white space is going to be exactly reconstructed your energy function is going to be flat equal to zero everywhere so the question now is how do you make

Self-Supervised Learning


sure the energy is high on points that you don't train on and this R of Z here is a regular razor that is going to make you pay for choosing a Z that's outside of kind of a small volume if you want okay and the usual trick something that gmer has worked on a lot is sparse coding so it's fast coding the the regular reservations the l1 norm of Z so it's the sum of the absolute values of

Self-Supervised Learning


the components with Z and the effect of this is basically to make the Machine want to make many components of the zero which is why it's called sparse coding it's you're trying to reconstruct a data point why as the product of a sparse vector with lots of zeros in it multiplied by a matrix okay so assuming W is known I'll give you a why you find the Z that minimizes the sum of those

Self-Supervised Learning


two terms it's going to give you a Z vector that is sparse has got lots of zeros okay and the region of space they have low energy other ones you know basically that energy once you minimize with respect to Z is the energy of every data point and because Z is constrained to be sparse it's going to be a small region of space that takes low energy and outside is going to be high energy

Self-Supervised Learning


okay so by adjusting the alpha coefficient here you can make the region of space that is properly we constructed as small as you want or or larger if you want the general form is something like this where Z equals some cost function that measures the discrepancy between the data point and the decoder function applied to Z the decoder function is trainable and then you have a

Self-Supervised Learning


regularizer that limits the information content essentially of z k-means does this implicitly by restricting Z to be a discrete variable okay PCA or things similar to this implicitly do this by limiting the dimension of Z ignore the bottom for now so if you apply sparse coding to this little spiral data set this is the energy function you get so every line that you you can you can sort of see

Self-Supervised Learning


here is a different linear subspace which is actually the selection of a different column or pair of columns of the W matrix and and the whole thing kind of fits the fits the data right okay the system is trained to just minimize the reconstruction error plus the rigor Iser on the data points and because it's got this regularizer as a consequence it gets high energy to stuff

Self-Supervised Learning


that is outside you apply this to em mist you get things like this so these are the columns of the W matrix and what sparse coding does in this case is that it reconstructs every digit in m nest as a linear combination of a small number of those guys okay because only a small number components of Z can be nonzero so a small number of columns will be selected so every digit is gonna be

Self-Supervised Learning


reconstructed as a linear combination of a small number of those you train the W matrix here in sparse coding but just with great understand you have to play a trick which is that you have to constrain the norm of those vectors to be within a sphere like less than 1 for example otherwise they blow up and the Z variable shrinks but that's not really interesting it's gonna be degenerate

Self-Supervised Learning


solution so you have to constrain the those vectors the columns of W to be small to be bounded and as a result when you train the system the the system identifies those pieces that need to be combined to form characters as kind of small pieces of strokes which is kind of a logical way of decomposing a character into elementary components some our colleagues at Facebook recently used one

Self-Supervised Learning


of those latent variable system but they kind of limited the information content of the little representation by kind of making it low dimensional they call that glow generation through latent optimization this is by version of skies around Lopes pass and shun and if you train on faces for example you get you know recon faces like this with you know relatively

Self-Supervised Learning


low dimensional latent vectors and you can sort of interpolate that latent space and sort of interpolate from you know one phase to another this is you know not as I mean this is a few years old but you know it's it's kind of at the time similar to the the Gans of the time there's been a lot of progress in Gans not so much in this particular approach but but it it's kind of

Self-Supervised Learning


interesting we're approaching the problem now there is an issue with sparse coding which is that if I give you a why you have to run an optimization algorithm to find the optimal Z that minimizes the energy that might be a little expensive even if you use Julian's code called spans it's still a little expensive so here's a trick and the trick is you're not going

Self-Supervised Learning


to make the Z variable a latent variable that you have to optimize over you're just going to make the output of an encoder so you're going to train a neural net here to predict what the optimal code is for sparse coding let's go as possible to encoder so you have a piece of data here you run into an encoder it predicts the value of a variable this variable is regular wise

Self-Supervised Learning


to be sparse through this RZ which could be another one norm and then you reconstruct the d adwords this is in the general form of regular risotto encoder in this particular form of an l1 regularizer that's called a sparse o2 encoder now these two forms of it a form where is just a neural net with an additional term in the in the objective function for training is another form

Self-Supervised Learning


where you still have Z as a written variable but now you have a third term in the energy that makes you pay for making z2 different from the output of the encoder and so the procedure is you give it a Y back within the sense you find a Z that minimizes the sum of three terms the reconstruction error the regularizer and the prediction error here which is the distance from Z to the

Self-Supervised Learning


output of the encoder which is a prediction of Z if you want then once you have that Z you do one step of gradient descent in the parameters of the encoder of the decoder so the decoder tries to get it put to get closer to why and the anchor get try to guess its I'd put the bar to get closer to the by minimizing this okay and there's couple papers on this that

Self-Supervised Learning


are about ten years old we used to call this predictive spas decomposition there's a form of equal lista so if you run this algorithm on natural image patches this is the running algorithm actually running and you start with random so what you see here each square is one column of the W matrix in the encoder I believe here but there is a similar one in the decoder

Self-Supervised Learning


which looks very similar and as learning proceeds of you train on more and more natural image patches what you see is a pattern of kind of features X appearing they end up being oriented edge oriented contour detectors you want something like this is a compositional form of this where the reconstruction now consists in taking a bunch of feature maps those are your latent variables

Self-Supervised Learning


convolving it with a bunch of kernels summing up the results and that's your reconstruction okay so instead of having scaler here you have future maps and instead of having columns of a matrix you have conditional kernels and you get really cool filters so those are the decoder filters these are the encoder filters for various number of various dimensions of the of the code here so

Self-Supervised Learning


get natural feature is emerging completely and you know unsupervised from training all natural image patches let me skip ahead a little bit okay so there is a formulation you've probably heard conservation electronic odor and when you look at those papers is a whole bunch of you know sort of vegetable bounds and all that stuff and it's very hard to understand intuitively

Self-Supervised Learning


what's going on so I'm going to kind of formulate this in the context of this energy based regularized o2 encoder model what a violet one color is and I'm sure for you is going to be enlightening if you haven't completely understood already with a version of the 20 color is so version auto encoder is an auto encoder you fit it the data a piece of data image patch or whatever running to

Self-Supervised Learning


an encoder you predict the code and then you add noise to that code in a particular way you add Gaussian noise to that code and then you run through a decoder and you have a reconstruction error and you minimize a reconstruction error by training this entire system now okay so in the space of codes in the z space every training sample is going to be a point okay and the house is going

Self-Supervised Learning


to be some sort of structure to that to those to those points you don't know if you add noise to each of those points which is what the factional encoded does you turn every single one of those points into a fuzzy ball right so you have a fuzzy sphere around every point now here's a problem if you give this data point and then you add noise and the noise puts it here when you

Self-Supervised Learning


reconstruct the system is going to think it was that point and so the reconstruction error is not going to be very good because it's going to confuse one you know one data point with another one so when you train the system the consequence of of this noise is that all those fuzzy balls are going to fly away from each other okay they're going to try to get as far from each other as

Self-Supervised Learning


possible to minimize the confusion and that doesn't help you it just makes the weights of the encoder larger but doesn't really help you so to prevent this from happening you're going to attach every every single one of those spheres with a spring it to the center okay so you tell us years you can't go too far you know you pay price for going too far and so the

Self-Supervised Learning


system is going to try to find a trade-off between you know pushing the balls far away from each other but not being able to do this is going to merge or or let some of the spheres interpenetrate as long as the reconstruction error that this causes is not too high and so in the end is going to try to find some sort of structure in the latent space that will sort of

Self-Supervised Learning


organize those points so that they you know basically capture the structure of the data and you can think of this as just another way of limiting the information content of the code so our regular riser RZ with sparse coding was limiting the information content of the code this is another way of limiting the information content of the code you can do it by imposing low dimension sparsity

Self-Supervised Learning


various other ways of this type or you can impose it by adding noise as long as you limit the norm of the other points now it's a bit more formulas in inversion autoencoder where the side of this of those balls is not fixed it can actually vary in all dimensions but there is a cost function that makes it makes you pay for making it significantly smaller than one and

Self-Supervised Learning


there's another term that makes sure the average the mean of all those points is actually centered on zero which is not with the spring analogy doesn't quite quite quite do you notice you go to encoder is this idea where you you take input data you corrupt it you ready to go to encoder and you reconstruct the original point without corruption and so visually it looks like this where this

Self-Supervised Learning


would be the data point every every one of those points is a is a a corrupted training sample if you want so it's still the spiral example and then you train a neural net to take you know you take a data point you corrupt it and then you train a neuron that to map from this to here this is the input this is the desired output very simple you transistor on that and that way you

Self-Supervised Learning


can plot is the blue points here are the outputs of that auto encoder neural net for every one of those golden points whatever right so I don't know this one maps to here and this one too there this one to here etc so you see that the system has kind of learned really it's going to similar representation here but you take every grid point and every point on the grid in the space and the

Self-Supervised Learning


blue points are the image of every single one of those points and they primarily cluster around the region of high data density which is what you want the energy function is just the square reconstruction error between the between every point in the space and the point that it maps to and those little vectors here can indicate the displacement if you want and the color indicates the

Self-Supervised Learning


energy function it's an issue which is that the energy is actually zero on this this is not a Ridge it's actually a valley and it's terrible okay so there's a it's a flaw of denoising auto-encoder it can create values in places that you shouldn't have won but it works really well for NLP so some of you may probably have heard of Bert okay it's gonna take on the world

Self-Supervised Learning


by storm in NLP and Bert is a special case of denoising auto-encoder where you give it a piece of text a window of a few hundred words and I know I'm at a time a window of a few hundred words umask so the the corruption consists in masking some of the words in that sentence or in that piece of text typically 15 percent of the words and then you train a giant neuron that

Self-Supervised Learning


transform a neural net which I'm not going to go into the explanation of of basically predicting the words that are missing and they're it's easy to handle the uncertainty in the prediction because at the output you have a big softmax that gives you a probability distribution over all words your dictionary so here I don't have the issue that you have with editing video

Self-Supervised Learning


where it's a high dimensional continuous space it's easy to handle uncertainty in discrete space which is why it works well in this case so you train the system to do reconstruction on tons of text billions of segments of text and in the end you use the internal representation learned by the network as input we to a downstream supervised task like you know natural language inference parsing

Self-Supervised Learning


winogradsky all kinds of stuff and this meets the record on just about it you know although the benchmarks in the glue so glue is in super glue our set of benchmarks in natural language understanding and systems based on bird the latest one is Roberta from Facebook actually has the record I think a few days ago Microsoft came up with an improvement on Roberta that actually can

Self-Supervised Learning


I brought the performance a little better it works really well doesn't work on images so if you use the same trick on images you block a piece of the image and you ask the system to reconstruct it doesn't quite work it doesn't give you features that are very useful in the end for for vision let's see I'm gonna stop here okay I just want to want to show you a

Self-Supervised Learning


simple a simple example of prediction and or certainty that that's that's interesting and it's an example of training one of those predictive model to be a forward model for a control problem of driving a car for example so if you drive a car you might want to be able to predict what the cars around you are going to do and it's not deterministic so if you are if you are

Self-Supervised Learning


this guy you have this car these are all the cars around you this is a little rectangle extracted around around yourself and it might be useful to predict what the cars around you are going to do if you want to plan a kind of trajectory that will avoid you know the probability of future accidents so one thing you can do is something like this where you take a few frames so you

Self-Supervised Learning


have the blue car the green cars our cars around you you're observing the world doing some stuff around you what you're going to train is one of those predicting model a big Commission on that with some latent variable to predict the next frame and the precise I'm going to go into the details with precise architecture it's basically very similar to a VA II in this case where

Self-Supervised Learning


you get the state of the world which is the environment of the car you run it to a couple you know few layers of a ComNet you add a latent variable here which goes through another couple layers of a neural net the certain variable is relatively low dimensional and then you run through a decoder to predict the next frame and to predict the value of the latent variable there is actually an

Self-Supervised Learning


encoder and of course the system could cheat because now it has the answer of the target and so you could just use the value of the target to predict the latent variable that wouldn't be very useful so you restrict information content in the latent variable because this looks like an auto encoder and you do this by the noise via e style essentially skipping details so what you

Self-Supervised Learning


get here is this is the kind of prediction you get if you don't use the latent variable you set it to zero all the time you can deterministic predictions but they become blurry there quickly after a while and those are multiple predictions that you get by making different samples of the latent variables right so you get different futures by getting different samples of

Self-Supervised Learning


the latent variable you can use this type of model as a forward model in a control system where you have the state of the world the action you take whether you turn the wheel without your accelerator brake you run this to your forward model gives you the next state it takes a random sample from the latent variable to make that prediction you can read this multiple time steps there's a

Self-Supervised Learning


cost function that measures if the car is in lane if it's too close to other cars it's differentiable so by back propagation you can propagate gradient to this entire thing and train another neuron that call a policy network to figure out what is the best action that should take so as to minimize the expected value of the cost and it's a it's not reinforcement running it's all

Self-Supervised Learning


differentiable so it's just back prop okay no no reinforcement there if you do this it doesn't work very well you have to regularize the system by making sure it stays in regions of the space where the ford model does a good prediction which are not going to go into the details off and then after you do this you run the the policy network so there's no planning necessary in the end

Self-Supervised Learning


the policy network has already kind of thought in his mind when it was training all the bad things that could happen and this is the the blue car here is driving itself in traffic and you have to realize that the traffic doesn't realize the car the blue car is here it doesn't see it is maybe a better example here so the yellow car is a real car the blue car started at the same location but

Self-Supervised Learning


In 2016, JAMA published research demonstrating the efficacy of a deep learning algorithm. We were able to train a deep learning neural network to recapitulate the majority decision of 7 or 8 US board certified ophthalmologists in the task of grading for a diabetic retinopathy. The type of deep learning algorithm used to detect diabetic retinopathy in that study is called a Convolutional Neural Network, or CNN.

Machine Learning For Medical Image Analysis - How It Works


CNNs enable computer systems to analyze and classify data. When applied to images, CNNs can recognize that an image shows a dog rather than a cat. They can recognize the dog whether it's a small part or a large part of the picture - size doesn't matter for this technique. It can also classify the dog by breed. CNN systems have also been developed to help clinicians do their work including selecting cellular elements


on pathological slides, correctly identifying the spatial orientation of chest radiographs, and, as Dr. Peng mentioned, automatically grading retinal images for diabetic retinopathy. So let's open the deep learning black box to understand how this works. First, a CNN is not one process. It's actually a complex network of interconnected processes, organized in layers.


With each layer, the CNN can detect higher-level, more abstract features. When the CNN is identifying these features, it uses something called a filter. Here's how Larry Carin, one of the authors of a JAMA Guide to Statistics and Methods article on CNNs, describes a filter: So, we think about a medical image, a medical image in radiology or ophthalmology or dermatology is characterized by local structure,


could be textures, it could be edges, it could be curves, corners, etc. And what these filters are doing are constituting little miniature versions of each of these little building blocks. And the way that the CNN looks for these building blocks is the C in CNN, and it stands for convolution. It's a mathematical operation that looks pretty complex. But, actually, it's very simple.


It's a very simple concept. It's kind of like you've got this filter, and you're walking to every part of the image, and you're just asking the question, how much does this image look like that filter? Think of it like this: you have a drawing, that's the image, and you have a stencil, that's the filter. You take that stencil and pass that stencil over that drawing that you have,


and as you do that you will see that some parts of the drawing become more visible than others as you do that, right? And that process of sliding that stencil across this drawing is essentially the process of convolution. Now that we've explained what a filter is and introduced the concept of convolution, let's use an analogy of written language to understand the relationship between the filters


and the hierarchical structure of the layers in a CNN. We will simplify the explanation by using an analogy. The analogy is a written document. In order to communicate through writing, we organize it as a series of paragraphs, which are composed of sentences, those sentences are composed of words, and the words of letters. So reading a document requires assessing the relationship of letters to one another


in increasing layers of complexity, which is a kind of "deep" hierarchy, like the hierarchy in image analysis. Continuing with our analogy, let's say we're looking for the phrase Ada Lovelace in a paragraph. Ada Lovelace was a mathematician and writer who lived in the 19th century. And she holds the honor of having published the very first algorithm intended to be used


by a machine to perform calculations, which makes her the first ever computer programmer. In the first layer of the network, a CNN looks for the basic building blocks of an image. The basic building blocks of written language are letters. So in this analogy, the filters the CNN uses in the first layer would be letters. Let's zoom in on the word "Ada." Here is what the convolution process would look like for the letter A. When the


"A" filter overlies the letter "A" in the original image, the convolution output would generate a strong signal. This signal would then be mapped onto something called a feature map. The feature map represents how well elements in the image align with the filter. If something is there, the signal outputs white. If nothing is there, the signal outputs black. CNNs generate a feature map for every filter.


So in our analogy, there would be a feature map for every letter. These feature maps would then become the input for the second layer. In this layer, the CNN would spatially align and "stack" all those maps from the previous layer. This would allow the CNN to then look for short, specific sequences of letters in all the feature maps simultaneously. So the CNN would use a new set of filters to look for specific letters that are adjacent


to one another in particular sequences. In our analogy, the second layer would look for places where the letters A, D, and A are in sequence together making the word "ADA". It would also look for places where letters A, C, E, L, O and V are adjacent to one another using filters for LOVE and LACE. The output of the second layer would be the feature maps for those three sequences of letters.


In other words, in those feature maps, strong signals would be present where the sequences ADA, LOVE and LACE are located in the original paragraph. In the third layer, the CNN would stack and align these three new maps and perform more convolutions-this time identifying where longer words and groups of words are located. So the CNN could at this point identify where in the original paragraph the sequences of letters


and words making the phrase "ADA LOVELACE" are located. In our analogy, we were looking for a phrase consisting of only two words. Had we been looking for a longer sentence or even a paragraph, the CNN would deal with the greater complexity by having more layers. We've omitted quite a few details about CNNs for simplicity, but this captures the essence of the model.


But what does this look like for actual images, like identifying diabetic retinopathy from an ocular photograph? Images are made out of pixels rather than letters. In a digital context, a pixel is the smallest, controllable unit of an image represented on a display. Each pixel is a representation of a tiny portion of the original image. Think about pixels like creating a drawing with dots where every dot has a color


value and an intensity. The more dots used, the clearer the image becomes. The filters a CNN uses in that first layer are small squares of pixels that correspond to things like textures, contrast between two colors, or edges. These are the image analysis-equivalents of the letters used in our analogy. And as a CNN goes up in the hierarchy, it looks for combinations of these filters,


getting more and more complex with each layer. As the complexity increases, the CNN gets closer to identifying what it's looking for. So the specific features analyzed at each layer help put the whole thing together. So, for example, some of the earlier work showed that some layers tend to be better at extracting, sort of like, edge-like information. Meaning that, for example, if you combine different kinds of horizontal edges,


we might get a continuous line that resembles the retinal blood vessels. And as you combine more of those and start to encode more higher-level concepts such as, you know, is there a micro-aneurysm here, is there bleeding over here, is there other lesions in the image? And right at the very end is where these, after these multiple layers, the network will try to then condense all of that information down into a final prediction.


In this case, severe diabetic retinopathy. Developing a CNN to help identify diabetic retinopathy was motivated because many patients with diabetes are not getting screened frequently enough. We have to screen diabetic patients once a year or we should, and there are some barriers to getting that done. Some of it is just, you know, not having enough trained professionals to do that task.


It's also not having that expertise available where the patient is. It's not that, you know, there aren't retina specialists in a metropolitan city four hours away, it's that there isn't a retina specialist at your grocery store. And CNNs could facilitate the integration of diabetic retinopathy and other screening programs into primary care. But before that happens, more research,


especially prospective clinical trials, are needed. The way we do approach these things is really the way that medicine usually works, which is to say, "let's do validations of the method again and again and again until we're sure, we're reasonably confident that it really works on many kinds of images, in many settings for, you know, many different patient populations." And


so from my perspective that's really at the end of the day what's most important: does it work on real patients and is it reliable? The excitement generated by early results has already spurred several research groups to look into the efficacy of CNNs in clinical practice, which could potentially finally get CNNs from the bench to the bedside. I think we're on the third or fourth technological revolution


where neural networks are coming to the forefront, and I really hope that this time we'll get it right. But there were failures in the past where people used the technology in suboptimal ways and we don' t want it to happen again. One has to make sure that we have appropriate and sufficient data for development, validation and testing, and that we're solving actual clinical problems.


At the end of the day, one thing to take away is that even if, as a clinician, it can be hard to understand exactly how a CNN arrives at its diagnosis, it can still be a useful tool. And this is similar to how many clinicians use other widely-adopted technologies. Consider antibodies: You know, as a clinician I may not know exactly where that part of an antibody kind of binds to, but I'm comfortable after looking at some


of this clinical validation of using Lucentis, for example, for an injection, right. This is kind of like any new breakthrough technology: needs validation and needs transparency, but I think, you know, the medical community in general responds very well to new technologies that have been validated. This video is meant to be an introduction into the topic of CNNs.


Suat CS 287 fall 2019 sorry to miss you last week you covered on Tuesday contacting variant optimization with Igor and Thursday I believe you covered motion planning with one of the TAS Harry there was a couple of slides that were uncovered at the end of last lecture I'm gonna let you study those on your own essentially there's a topic of lqr trees it's not gonna come in the

Lecture 11 Probability Review, Bayes Filters, Gaussians -- CS287-FA19 Advanced Robotics


homework but it's quite interesting it's a way to combine motion planning with lqr to have a more efficient motion planner effectively because motion planning tends to be very local wind sampling based lqr is a not that local it has a basin of Attraction and so combining both of those into lqr trees is pretty interesting and then the other thing you didn't cover was shortcutting


typically when you use a sampling based motion planner you will at the end of a path that's very jagged short cutting is the idea that you just check if any of the steps along the way you can just skip because it's just a little bit of a detour and use a straight line instead of course depends on the dynamics of your system if your system has peculiar dynamics you cannot just do straight


lines but if your system is able to follow any path except when there are obstacles thing you can do straight lines if there's no obstacle so that's shortcutting is a little more detail in the slides and often actually you also run non-linear optimization for control actually at the end to smooth out your path and get a locally optimal path so those are the highlights of what you


didn't cover but we're gonna not cover in more detail but the slides have the detail so they will switch to sometimes the second part of the course first part of the course was all about finding optimal actions how do we given a dynamical system some environment really find a sequence of actions or a policy that optimizes expected reward now we're going to look at the complementary part


which is trying to make sense of our sensor readings such that we actually even know what the state of the environment might be and then act against that and we'll need a lot more probability here so we'll start with a bit probability review then we'll look at Bayes filters which are going to be the core of what we'll be doing and we'll look at base filters today in the


simplest setting which is just discrete state discrete observation and then we'll look at gaussians which will allow us next lecture to do it in continuous state spaces and in many ways similar to how lqr give us local solutions similarly with Gaussian siva for nonlinear systems we'll be able to find local approximations to the probability distributions all right


any questions all right then let's get started so why might we care about probability in robotics well often the state of the robot and the state of its environment are unknown and only noisy sensors are available so we don't have a sensor that just says here is the state instead you measure something else probability provides a framework to fuse that sensor information to a reasonable


estimate of what the state might be and so the result of what we would compute would be a distribution over possible states that the environment and robot could be in there's another reason we care about probability which is something we've already covered a little bit which is that the dynamic is often stochastic so we can optimize for a particular outcome but the only optimize


obtain a good distribution of our outcomes and again probability provides as a framework to deal with that we've done that in simple settings so far but we'll expand that later and we'll actually bring both of those together in a future lecture where we'll look at the notion that the actions you take could be taken to reduce uncertainty about the world so you actively go seek out sensor


information that could help you understand better what the state of the world is and then collect more reward formulas are called pom the peace partial observable Markov decision processes and that will be somewhat the culmination of bringing together what we're covering in the next few lectures and what we've covered in everything so far let's look at an example how


after what would be the state position of the helicopter orientation velocity angular rate what would be the sensors you have you don't have direct access to position orientation velocity angular rate you have access to some of those in a noisy way and then others you don't have access at all so GPS gives you a noisy estimate of position sometimes also velocity typically only up to a


couple meters accuracy so not super precise but it's a noisy estimate of position typically you put the inertial sensing on your robot so for helicopter maybe you have a three axis gyro so charr is a angular rate sensor so it gives you three three numbers at any given time measuring the angular velocity around the three main axes of the helicopter if that's you know if you


mounted axis aligned then three axis accelerometer accelerometers measure acceleration but not exact acceleration it's a little trickier than that Adashi measures all zeros in freefall so if you're free falling an accelerometer measures all zeros and anything that you do that's not free-falling for example if you're standing on the ground your accelerometer will measure 9.81 m/s^2


opposite to essentially gravity because you're resisting gravity and that's what it's measuring at that time so in some sense accelerometer is not not just an in fact much more accelerometer is about measuring orientation very often than it is about measuring your actual acceleration because any resistance of gravity you'll be able to measure and from that get in some sense especially


if we're on the ground understand where gravity is pointing which gives you a lot of information about your orientation then three axis magnetometer which well what is it magnetometer it measures the earth magnetic field so earth magnetic field you might think of it as North but it's actually if you measure it in 3d it's not really north it's roughly north but it actually also


points into Earth but it's known which direction at point since you are measuring in the frame of your sensor where is the magnetic field pointing and that gives you information about the orientation of the system that you have some tricky things that of course if you have something that generates its own magnetic field like maybe have some magnets on your system where you have


some high currents that induce magnetic fields then it might perturb those measurements and not really get the earth magnetic field but something else are you flying near a power line or something it might perturb the measurements you're getting dynamics there is noise from the wind there's unball dynamics and engine in the servo motors and in the blades of the helicopter so overall we don't


really have access to state directly but we have things that relate pretty closely to state and we should be able to distill from that a likely estimate of the state or a distribution around some mode maybe of what the state might be how about a mobile robot inside a building the state could be position and heading is let's say a slow robot is just slowly moving you don't care about


velocity just the position in the direction it's facing then sense source odometry which is sensing motion of the actuators for example wheel encoders so on your wheels you might measure how much your wheel has turned if you didn't know the diameter of your wheel you know how much you have moved well that's approximately true because the wheel might have slipped there might have been


bumps in the road where it's if it's not flat you don't move as far forward as you thought you would have because you really have been going down and back up but it gives you some kind of measurement of how much you've moved then often people with a laser rangefinder on a mobile robot why is that it sends out a laser beam and then it measures how long it takes for that


beam to come back and that gives you a measurement of twice the distance to the obstacle in that direction and so it's a nice way to directly measure 3d it's a little bit problematic at times because sometimes you have mirrors and then the thing doesn't come back your reflects off in another way or you might have glass that acts much like a mirror and the same thing will happen you don't get


the measurements back but if you have nice matte surfaces it's a really nice way to measure how far away things are dynamics is noisy because of wheel slippage you might say my will turn this much but what if it slipped and you don't know what your next state is going to be if you don't know how much it slipped and it could be unmad unmodeled variations in the floor that effect


essentially where you end up all right so those are two examples just to motivate why we might want to deal with probability distributions in that yeah if this is what we're faced with or the helicopters what we're faced with we're now going to have a deterministic notion of this is the state we're just going to have a bunch of sensor information and the result that best


will be a distribution over possible States okay so today what we're going to do we're going to a probability review hopefully that's a review if what we're doing the probability review section is not review you should go study to make sure that it feels like review as soon as possible then we'll do Bayes filters where we will look at the foundation of a lot what we'll cover in the discrete


setting and then we'll start looking at gaussians which will form the foundation for what we'll do a next lecture to do it in continuous space all right so probability theory as a lot of math it starts with some actions and and everything follows from that so probability theory actions here is that you have some outcome a and it could be many possible outcomes an outcome a the


probability of the outcome is assigned a number between 0 and 1 maybe you know I don't know the probability of being in the correct room for lecture is some probability assigned to it then the probability of the union of all possible outcomes Omega is essentially everything that could possibly happen the probability of everything that could possibly happen something in that said


to have happened should be 1 otherwise that's not the proper Omega that captures everything and that also means nothing can have a number sign higher than 1 the probabilities don't go above 1 and then probability of the empty set well the empty set is never containing the outcome that happened whatever happened is something it's not in the empty set and so probability of that is


0 and then the main thing here is essentially how you kind of keep track of probability the probability of the union of two possible outcomes a and B is a probability of a plus probability of B but it's possible that a and B happen at the same time and then you're double counting down as you subtract back out the probability that a and B happened at the same time which is a intersection B


and that gives you the probability of the Union pictorially looks like this you have your Omega space which has all possible events in it then there is outcomes that make a true there is outcomes that make be true and then there's outcomes to make a and B through the intersection and the probability of a union B is probability of a plus probability of B minus probability of


the intersection one abstract or abstract or concrete depending on your mindset way to think of it is that everything that can possibly happen is a point in this rectangle pictorially that's like every possible state the world could be in is a point in that rectangle I mean we might never know the exact details of it but every possible state is a point in that rectangle and


then a is a property of the state of the world that holds true for all the ones that line that blue circle the bees have property of the state that holds true for all the states of the world in the yellow circle and then a intersection B is a property of the world that holds true for all the points that are in the intersection and so you can think of a and B just as properties or descriptions


of you know abstractions of the world of the state whereas individual points in that rectangle are in some sense the full state of the world which are probably never observed will never really talk about because we don't care about all the details we'll talk about a and B but in terms of how it's all set up that's an easy way to think of it then when you use these actions you can


come to new conclusions I mean that that's kind of how math works you posit some actions in the you do some derivations and you have a new thing that's useful to use so for example probability of a union on Vega minus a well on the Left we just look at what's in the parentheses is well a union with omega minus 8 that's omega so the probability of Omega is 1 that's what we


derived up to here on the right-hand side we use that thing that's a probability of Union is promotive each individual event- probability of the intersection that's this we've worked that out then we know that this probability is zero and now we have the probability of a plus probability of the complement of a has to be equal to one and so that's the kind of properties you would derive all


right so there are discrete random variables and continuous random variables so here we have Omega again the rectangle and again think of every point in the rectangle as representing one possible state of the entire world and then X is our random variable we work with X can take on four values if the event in indicates the world is in a state that's in that leftmost rectangle


we say x equals x1 if it's in that triangle above it's x equals x2 and so forth and so there's only four values X can take on the probability associated with X equal X 1 we can think of like essentially the mass associated with that region in the rectangle and we call P a probability mass function a simpler example than the abstract rectangle there would be a coin flip heads and


tails probability half for each that's another example of a distribution again you can think of as in the context of that rectangle though you can say I have a rectangle I split it in half there is heads and tails and you know have the states of the entire world my coin came out and heads have the states of the entire world my cone came out tails we're ignoring everything else about the


world we're just looking at heads versus tails and not the details of everything else in the world continuous random variables X takes on a value and some continuum you cannot now associate the probability with X taking on a specific value because if it's infinitely many things it's a continuum if you want to assign finite probability to think so it'll sum up to something higher than 1


which we can have it needs to be something to 1 so we can so many more rash you have to integrate now so we'll do is this a probability density function is defined saying that if you look at mass under the curve so the probability for X being an interval A to B is the integral from A to B of P of X DX for example if we think from zero here maybe till this point we could take the probability


Mouse and that's then the in the area under the curve would be denoting how much parabola we associate with X landing in that interval so again we can't really talk about X stick on a specific value here the best we can do is say probability of X lying in an interval you could say X probability of X taking on a specific value would say it's zero but then to have something


meaningful you take intervals and assign probabilities to the intervals any questions so far all right then the most important thing and what we'll be working on is to look at distributions that involve multiple variables so what why is that well the reason is that has already talked about we'll have things like real state that we care about and sensory measurements


so I want to somehow talk about the joint distribution between the two so if we measure one of them we can say something about the other one or we'll deal with dynamics of the world and so we want to relate the distribution now with the distribution at the next time and so both of these involved joint distributions over two random variables rather than just looking at a single


variable all right so the joint distribution over x and y is denoted this way then in simple scenarios simple scenarios X&Y might be independent meaning of the probability of x and y taking on specific value small X small Y is a probability of X taking on value x and y taking on value of y that's the case actually knowing X does not tell you anything about Y or


the other way around so it's actually not so interesting case mathematically simplifies things but yeah if you had a sensor where your sensor is independent of the state of your system the sensor cannot inform you about the system sometimes the property that holds true though and then it's it's good to know and good to take advantage of mathematically but you don't want to


build your systems that way necessarily then X given Y is the probability of X taking on value of small X given Y has value small Y so what we have here is that as a definition we write it as X given Y is the probability of x and y divided by probability of Y so it think again about that rectangle there's a region where Y takes on the values small Y and that has a certain surface and


that's py and we look at the region where X takes on value is small X within that region that's B X comma Y and then the fraction of the surface it takes up that's the conditional we can also rewrite this as the probability of X comma Y is the probability of Y times probability of x given Y this will become very useful many many situations because let's say we know the


distribution for Y and want to know something about X which might be the distribution at the next time slice and if we notice in about X given Y this may be the dynamics of the system to go from Y to the next time X then this equation will tell something about the joint and hence tell us something about X then if x and y are independent then the conditional of X given Y is the same as


the marginal for X so P of X will call the Marshal for X and so that means that sometimes in our math we'll be able to simplify things you have an assumption X in planet of Y we see X given Y appear we can just get rid of the Y and simplify to just px all of this is also true for probability densities it's just in densities often it's a small P but other than that you can write the exact


same math and I will say something about the densities instead of the actual probability mass now let's work through the equations we'll be using the most all right so there are essentially two equations there are two equations we'll be using a lot and so let's explicitly step through them and see what they are one s cold law of total probability was it say it says that and we'll write


it for the discrete case the probability distribution for X is equal to so probability x equals x is sum over all values Y can take on of P X comma Y so the reason that will be so important is because often we'll want to get rid of some variables and move to another variable and the way we're going to be doing that is by constructing a Joint Distribution over those two variables


somehow and then something out the 1 we don't want anymore more specifically the way this will typically be used is here we'll say well PX is equal to sum over y and imagine we already had access to the distribution for y and we need to construct a joint because you want to bring in X now we might have a model for how X is related to Y so you'll have px given Y and so this equation here is one


that we'll find ourselves using a lot one of the two equations we'll be using over and over same thing can of course be done for densities and densities this summation would become an integral because you have to integrate over all continuous barrels values the variable can take on whereas in some in discrete variables you just sum otherwise same thing though in general whenever there's


a summation you can put an integral same math we'll go through so that's one of the things we'll use a lot the other thing we'll use a lot is Bayes rule Bayes rule hopefully sounds familiar where how do you derive it we actually derive it from looking at this the expression for the joint the X comma Y is px given Y x py which is also because the roles of X and y are arbitrary it


doesn't matter which one comes first or second in this equation here we can swap the roles and we have P Y given x times PX from that using just this part over here we can write P X given Y as P y given x times px over py let's interpret this equation for a moment so what are we doing here we're interested in finding this mission over X given Y so we can associate a story with this


imagine X is the state of the system and Y is a sensor reading we might obtain and so we want to know what's the distribution of our possible states of this system given we have a sensory reading Y well it's not easy to come up with a table for that it's not easy to just say oh you know what I'm just gonna build a table for a px given y because it's actually not really how nature works the


way nature works is that if there is a state that state causes the readings so this model here is really the causal model that we have available given a state it will cause a distribution over readings and when you build a sensor and you try to sell to people this is your sensor model that you would provide it said this is the distribution of our outcomes given this situation this is


fishing-rod come given this other situation and so forth for example for a laser range file you might say well if there's a map surface at a certain distance the reading you'll get back is within maybe two centimeters like maybe with denigration 2 centimeters around the proper distance reading something like that so that thing is your calibrated sensor and that you can you can get a


distribution for but what actually ugly will be distribution over state given that reading will depend on the prior X what you thought ahead of time might be likely states of your system and so they combined together and then of course is a normalization to make sure things sum to one that's old is this and often will write this as just 1 over Z times P Y given x times px or it turns out in a


probabilistic robotics book notation they love to use it I so often write as a dot x py given x times PX so we don't have too much worry about the thing at the bottom that's just a normalization we just want to understand a thing at the top which brings together the thing for which we are able to build a model which is the relationship from X to Y and how we can then use it assuming we


have a prior over X use it to get the distribution we really want which is distribution over state any questions about this these two equations are essentially the ones we'll just use everywhere and what we'll do today next lecture lecture after that if you fully understand those you should be in really good shape now we can have a new version of this so law of total


probability again but with conditioning what does that mean for both of these imagine for the first one here imagine we had already something to conditioned on maybe there was a measurement in the past that was already there like we really just in not going just from Y to X but there's some Z that's already present somewhere so what can we do well we can say okay P X


given Z is equal to we write the same equation sum over Y P Y times P X given Y but since we conditioned on C we need to ever wear a condition on Z and if you're ready I mean notation wise condition on Y and Z is just Y comma Z you don't need to draw another bar and so what we see happen is done the exact same equation can be written with just additional conditioning on Z or any


number of variables as long as you consistently put them everywhere so you can always add more conditioning and so often that's the version we'll be using this will have many things we've already observed in the past we want to conditioned on and then we'll want to apply the equation the left law of total probability and we'll just carry along everything so do our math


as if it doesn't exist we just need to consistently carry it along very straightforward got to be careful you can't drop it anywhere same thing is true with Bayes rule what if you have multiple things you are conditioning on a already condition of something else no problem Bayes rule with conditioning and the reason this should really be a surprise is you go back to the original


picture of probability right then you we have this rectangle and the rectangle is really the foundation of everything with a point in the rectangle corresponding to the full world state and then regions correspond to values the random variable takes on if you condition on something essentially we're just redefining the rectangle as being only only let's say this part that's our new rectangle and


that's the only part of the world we're working with possible States you can take on and once we've done that we put all probability on this thing we kind of renormalize it everything's there that's our new rectangle and the maths not going to change because we're just looking at this sub part of the whole rectangle but of course you need to do it consistently every step along the way


we need to consistently look at that sub part where we are having Z this would be Z equals Z we need to keep it consistently to that part we can sometimes have it sometimes not when we do this but then same thing with Bayes rule yeah initially it was for the full rectangle now we restricted attention to some sub region no problem we just condition everywhere


P X given Y comma Z is equal to let's it's equal to well what do we have we had Y given X but of course also Z because Z needs to be in the back everywhere now then P X but we need to carry along the Z and then normalize by Y again given Z but again the bottom here doesn't really matter much in baseball we usually know that we just need to compute the top part for every


possible value of x and once we computed the top part for every value of x we can see what it sums to and whatever it sums to is what goes in the bottom right and so applying Bayes rule is about doing this for all X and then summing to know what the bottom is and so here we see that we can incorporate new evidence we already had evidence we had already had X given Z we had a past observation


distribution over x given that observation then a new thing came in for Y and we have a model of how Y acts given X and Z and practice often the Z will disappear here because Y will only depend on X not on Z but in channel YK depend on X and Z and then we can combine that together to know what now new distribution is over state X then another concept that will be very


important is conditional independence so conditional independence is a lot like independence but a little less strict independence is a property done and sometimes it's nice because it simplifies the math whenever you run into it but if your variables are all independent then you can't really infer anything from one variable about the other variable and so ultimately you get no interesting


results out you need the variables to interact to get something interesting out now it could be that they interact but in limited ways and that's what conditional independence tries to capture so condition dependence we have X conditionally independent of Y given Z if and only if we have P X comma Y given Z equals P X given Z times P Y given Z so it's like independence the joint


between x and y is the product of marginal for X marginal for y exit that it's only true when you condition on Z once you know z x and y are independent but as long as you don't know z they might have a relationship for example z might be something I don't know like Z could be the weather and X might be is somebody carrying an umbrella and Y might be is the is the pavement wet or


something and once you know it's raining you you don't need to you already know that the rain will cause x and y independently it's not related directly whether pavement is wet and somebody cares umbrella it's through the fact that it rains which is the common cause for both of them and said I'll make x and y independent given see another example and the one that we'll see most


often in this class is if you know the state of the world then every sensor for every sense you're reading might be independent very often because if your sensor just acts on the real state of the world then the reading that happens first very happens next will be unrelated and we'll be able to simplify things okay so this is this is kind of really everything we need so we just need three


things we need law of total probability the regular version and the conditioned version we need Bayes rule the regular version the conditioned version and we need a notion of conditional independence which will allow us to simplify equations this is also equivalent to writing and also we'll see it that way equivalent to px given Z and Y but once you know Z X is independent of Y being


equal to px given Z and it's also equivalent to P Y given Z comma X being equal to Y given Z this once you know Z X does not tell you anything about Y now to be fair this condition the dependence assumptions that will be making sometimes in the real world might not exactly be true ever but they'll often be reasonable approximations and we might be willing to make him because


they drastically simplify the math and the competition we need to do to then run the algorithm in practice so often often these things are assumptions we make to approximate things that are almost independent in the real world questions so far yes mm-hmm so typically the way you would do it is this typically what you'd be given is from a previous calculation maybe or


just given would be this thing here which is the prior over X of PI over X given Z you'd be given a model of how X and Z in this case are just X in that case cos Y that's your sensor calibration model for example and that together allows you to compute this product for every possible value of X because we're conditioning on Z a favorite possible value of X yeah we're


computers for every possible value of X because Y will be measured in this scenario Z will be measured so y and z will be numbers available to us x will not be known can take on many values and so we'll compute this for every value of x now after we've computed this for every value of x we'll have a table of all these four values of X and those entries if we ignore the bottom will not


sum to one but we know it's supposed to be a distribution over x given y and z so then we know I also know we we ignored the thing at the bottom that does not depend on X that's key here there's no X in here so what we're computing is a table with entries that depend on X every X will have its own entry and we know all of them are missing the same rescaling which we've ignored and we but we can


find the rescaling by remembering that actually every all the entries in the table have to sum to one and so summing all the entries in that table will actually compute this thing over here mathematically we know that's a way to compute this and practice we don't often pay attention to what it really means we just say hey we want this we know this thing here does not depend on X so we


can ignore it as long as at the end we do a rescaling of all entries with the same scaling you cannot rescale one entry for x1 by the other one the other you need a fixed rescaling across all entries that you computed so the things written on the board we also have on the slides where often we have a pairing between the discrete version where there's a summation and


continuous version where it just replaced out some sign with a little squiggly thing and then you put a D something is over the variable instead of putting it under so essentially just always a place sum over the variable with this thing here and then a dy and then you got the same thing just for continuous space Bayes rule the terms we didn't name them but essentially often


the X the P of X is called the prior because we have it ahead of time Y given X is the likelihood of a measurement Y given this state is x is often what it ends up being solvent called likelihood and evidence is yeah that's just Y is what you measure to is a probability of getting that evidence this we just talked about when you do these calculations often you effectively