semantically relevant in downstream tasks because they have performed very very well so how does the Skip Graham model work how does how does it assign vectors to each to each word so first of all it has a dictionary so there is a word an input word and for each word you have a big dictionary and the diction dictionary is basically says that you know - the word - is going to be mapped to this vector point one Dada Dada Dada and so on the word learn is going to be mapped to that vector and then you also have these output vectors right here and


what you're trying to do is you're trying to take a phrase from the data set like this one right here and you take out one word like this word vector right here and you're trying to frame this as a prediction task so you're trying to frame this as in this case four different prediction tasks so you're telling your machine I give you the word vector and which other words are around the word vector you just tell it that you don't tell it anything else you just say which other words are around the world vector and the correct

[Classic] Word2Vec: Distributed Representations of Words and Phrases and their Compositionality


answers in this case would be to learn word and representations so these you construct for different training examples where you have an x and a y so the X is always vector and the Y is 2 and then the next training sample the X is vector and the Y is learn and and so on ok so this here each training sample is a classification task right and the classification task is and is as you can see no you can't see right here but the classification task is you have the input word and you classify it into one of many many many many many many classes


namely there are as many classes as you have words in the dictionary so each word in the dictionary will have a class associated with it right so an image nets you have like a thousand classes but in these and that's already a lot but in these tasks you're gonna have a hundred thousand classes because there are a hundred thousand words in the English language that you wanna want to treat and there are many more but in this case they leave away all the words that appear less than five times in their corpus that's still a lot of words


so it's like a super duper duper lot of a classification task but ultimately if you do something like this then the originally so the representation that you end up with is going to be very very good at doing these kind of downstream tasks and that's what they discovered so their skip gram model is nothing else than taking a word in predicting the surrounding words from that word and this is what it means this is the formal statement of the skip gram objective what you want to do is the objective of the skip gram model is to maximize the


average log probability this one so for the word we're considering the word t we want to maximize the log probability of each word W that is in around the word C I'm sorry around the word W in a context window of C that's exactly what we did before we take a word like this model right here and from it we predict all of the words around it in energy in a given window right that's all that's the entire objective and that will give you very good representations and this is how you would implement that so what you'll have is you'll have these vector


representation V that comes from your original dictionary those are the things you learn and then because you have like an 30,000 way classifier you know that a classification layer is nothing else than a linear layer followed by a soft max operation and that linear layer also has parameters these are the the Prime's okay so first you have the look up in the dictionary for the word vector right here and this is the vector of the classification layer now there are modifications where you can use like the same vectors and so on or you can also


make use of these vectors but ultimately you care about these vectors right here and the vectors here are simply the classification layers weights so here you can see that there is what you're trying to maximize is the inner product between the word that you're considering and the words around that word and you're trying to do a classification task so you need to normalize now this is the normalization constant and it goes over all of your vocabulary so that's what they tackle here they say W is the number of words


in the vocabulary this formulation is impractical because the cost of computing the gradient is proportional to W which is often large and that's 10 to the 5 to 10 to the 7 terms so many 10 like tens of millions of terms in your vocabulary that's just not feasible right so people have been you know sort of trying different ways to get around very very large number of classes and here it seems that that is really our bottleneck in the previous paper they've already shown that this objective can give you very good word representation


but now we need to get around the fact that we have such large vocabularies so the first idea here is hierarchical softmax and this is kind of a tangent i find this paper by the way it's sort of hard to read because it's like a half engineering paper but yeah so first I introduced this hierarchical softmax which is kind of a a distraction it's kind of a here is what we do here is what we considered first but then didn't end up using really they do compare with it but the flow of text is sort of that you expect this to be part of the final


model which it isn't so in the hierarchical softmax what you do instead of having this giant multi-class classification task right here you take all of these classes right here and you put them in a sort of a tree okay so you take this and you put them into a tree so instead of classifying you know let's say we have a thousand classes instead of classifying a thousand ways we first classify in two ways and then we classify in two ways again from each one and then we classify in two ways again as you know a thousand is like two to


the ten so we need approximately ten layers of this before we are actually arriving at a thousand classes but it also means that we only have to weigh classifications each time so in the hierarchical softmax we build trees like this and then we so we have a word we look up its vector sorry its vector and then we classify it for each of these nodes so your output isn't going to be a thousand a thousand log probabilities your output is going to be a log probability a binary log probability for each of the nodes right here so you want to know


okay here is it in the upper half or the lower half of my classes okay cool it's in the upper half okay here is in the upper half or the lower half and so on and you learn all to predict all of these junctions right here and that's going to end up you with you having to predict it less now of course you are constrained you impose a very big prior on the class distribution classes are an independently anymore namely if two classes here are in the same subtree that means that they are going to be predicted their predictions are going to


be correlated because the path to them is the same partially so how you arrange the classes here is very important and there has been a lot of work in this but as I said this is rather a distraction right here hierarchical softmax is a way to solve this however they went with a different way right here they went with this approach called negative sampling negative sampling has been it's been very influential not only in word to vecna --get of sampling is one of the cornerstones of the current trend in self supervised learning in contrastive


estimation and so on so this all of this you know it pops up in unlikely ways in other fields and it sort of I'm not gonna say it originated here but definitely it was introduced into the popular deep learning world right here so they say an alternative to hierarchical softmax is noise contrastive estimation okay so in noise contrastive estimation posits that a good model should be able to differentiate data from noise by means of logistic regression you know that seems very reasonable this is similar to the hinge loss and so on yada yada


while NCE can be shown to approximately maximize the log probability of the softmax the Skip current model is only concerned with learning high-quality vector representations so we are free to simplify in all these contrastive estimation as long as the vector representations retain their quality we define negative sampling by this following objective so this is very interesting they see okay noise contrastive estimation you know it approximately maximizes the law of probability so the noise contrastive estimation would actually be the correct


way to approximate their problem however they say well as long as you know as long as something reasonable comes out we're free to change that up a bit so they go with this negative sampling approach right here and you can see that this is this is almost the same so it's written a bit differently from the original softmax thing because the original softmax thing was written as a fraction and here it's as a sum but what you're trying to do in the noise contain the negative sampling framework is you trying to maximize the following


you're trying to maximize the inner product of the word you're considering and the words around them okay so if you're trying to still predict the words around you but now instead of having this prediction soft max over all of the classes you only have the soft max over a subset of classes so what you'll do is use sample words from your vocabulary at random and you sample K of them and you're simply trying to now minimize the inner product between those words and your okay so what is that ultimately lead to it ultimately leads to the following you


have a word like this word here- and what you're trying to do is you're not trying that much to predict the word sampling what you're trying to do is you're trying to say that in my space right here I simply want sampling to be closer than any other words that's not in the context window okay so so here is my word negative and here is my word sampling and I want these two to be close and if I if I sample another word like here this is the word cake if I sorry if I sample that I simply want that to be far away further than than


the word sampling okay so this is now a comparative it's not I classify sampling as the highest class it's simply I want to classify the word sampling against the other classes higher all right so and this is now much much easier so instead of a thousand or ten thousand or a million weigh classification and now maybe have I have a k plus one way classification right pretty easy right I simply sample K other words and as I assumed because I have so many words chances that I actually sample one that's in my context window is very


small right so I simply sample other words and I say well these other words are random they have nothing to do with the current frame that I'm looking at so they should be you know they can be whatever they want but at least they should be farther away than the words that are actually in my in my context and that is negative sampling the process of sampling negatives this right here and then making sure that the positives which are these here in this case the words in the context are classified with a higher probability


than the negatives for a given in right this here is the input word that's it that's negative sampling and of course yeah as I said you recognize this from current things like yourself supervised learning where you wanna have the same image augmented twice go through the pipeline you know you augment you put a little bit of different noise and then you have a different image and at the end you say these two should be close together well this other one should be far apart it's the exact same thing here except that


you have a different way of obtaining the positive and the negative samples in this case positive samples are everything that's in the context negative samples are just randomly sampled from the data set and that you know works of course that works much much much faster and you can see that this this turns out to give you vectors that are pretty good and you can train with higher vectors sorry with higher dimensional vectors you can train with bigger vocabularies with this this has turned out to be very very influential


as I said now with the rise of Burke and so on work to back is kind of getting forgotten but this was a revolution and distributed vectors so it wasn't the thing really it kind of was a thing before that but it wasn't really a thing that people used what people would do is still they would do n gram models before that so they would kind of diss diss they would sort of chunk up their sentences into engrams into overlapping engrams and then have a big giant table further where they index their engrams so the word I don't know so the word


hello is ID 1 the word hello there is ID 2 and so on so you have a big table for all the engrams and then what we would try to do is you would try to do this kind of bag of words estimation where you would take a you know whatever engrams appeared in sentence and you would have this big you know classification where you'd associate the engrams with each other and so on so distributed word representations were kind of a revolution at that point especially distributed representation that actually outperformed these old Engram methods so


there are a number of tricks right here that are I think not understood until this day for example the question is how do you sample these negative samples right right here this basically says get K words from your vocabulary at random according to this distribution right here now how are you going to do that basically you have a spectrum of options the one side of the spectrum is going to be completely uniform okay we sample each word with the same probability and the other side of the spectrum is something like sample this according to


their unique gram these are two different things they're they're opposites in this in this fashion so here you say hey some words appear way way way more often than other words shouldn't we prefer them when we sample right shouldn't we if we have a corpus and shouldn't we sample from the corpus and if in the corpus one word appears 50 times more than the other word then shouldn't we sample that 50 times more as a negative because it's you know so abundant and it should read get a higher classification accuracy


whereas on the other hand you could say no no we should simply sample every word in our dictionary uniformly they came up with something in between which they say both NC and negative sampling have noise distribution as a free parameter we investigated a number of choices and found that the unigram distribution raised to the 3/4 power ie unigram to death recorder outperformed significantly the unigram and uniform distributions for both NC and neg on every task which including language modeling this I think is a mystery until today and it actually


turned out that this exponent right here is magically much better than like the exponent of one or even the exponent of one half like you might be reasonably assumed that the square root you know might be something but the 3/4 I think turned out to be very good and very mystical so what does it what does it mean it means that you have kind of a balance between words that appear often in words that don't appear often usually in these kind of things you have a power law where we have very few words that appear very often and then you have okay


that's the tail shouldn't go up but you have a very long tail of words right and what you want to do is in this case you want to sample these words here more but they are they appear so much more often than if you simply sample according to their unigram distribution you basically not regard these words right here you'll forget about them and your performance will suffer because they do appear every now and then so what you want to do is you want to push the dose down a little bit and the optimal amount for the


little bit turns out to be to raise it the you raise it to the 3/4 strange but you know turned out to work well the other thing they do is they do the they do a sub sampling of frequent words so again this is a way to kind of push down the often appearing words where they say the most frequent words can easily occur hundreds of millions of times like in the array such words usually provide less information value than the rare words for example while the Skip current model benefits from observing the co-occurrences of France and Paris it


benefits much less from observing the frequent co-occurrences of France and the as nearly every word Co occurs frequently with in a sentence with the so they do another trick here to counter this imbalance between rare and frequent words use a simple subsampling approach each word in the training set is discarded with probably I computed by that formula right so therefore formula right here and might be asking again why why this formula so this is the sampling probability of a word and it goes with 1 over T T is a


temperature parameter and F is the frequency with which the word appears in the corpus so as you can see as the word appears more in the in the corpus then so this is the frequency as the word appears more then this thing goes down then this thing goes up so it's discarded with this probability so it's discarded with a higher probability if it appears more often where F is frequency if the word T is a throat T is a chosen threshold which shows this subsampling formula because it aggressively sub-samples words whose


frequency is greater than T while preserving the ranking of the frequencies although this subsampling formula was chosen heuristically we found it to work well in practice it accelerates learning and even significantly improves the accuracy of the learn vectors of the rare words as will be shown in the following sections so again something sort of arbitrary it's even it's more understandable than the 3/4 but still it's sort of arbitrary they experimented around they found this works well and then every everybody


ended up you know using that so that's how this kind of stuff happens ok so now we get into the empirical results and the empirical results in this case we're already sort of given in the previous paper but here they have these the analogical reasoning task where you can see that the negative sampling did outperform the others by quite a bit right here so the negative sampling approaches outperformed the hierarchical softmax and the noise contrastive estimation and in the previous paper they also compared with other baselines and saw that it


also outperforms those while being right time time-efficient so you can see that especially with these subsampling approaches the time here there's 36 minutes for and they again I think they have like a huge corpus that they train on these were to vac code turned out to be really really efficient code and that's why I got so popular as well they did the same thing for phrases right here so for phrases like New York Times and so on but this was kind of more of a this was more of a side thing the phrase vectors turned out to be you


know rather a side thing from the actual code right here so yeah as as I said this paper is very different from other research papers and that it's it's sort of half an engineering paper and all of these papers are they're kinda hard to read because they just kind of state some things in the order is kind of weird sometimes why they do things is kind of weird sometimes but you can't you know you can't deny that it had the quite the effect on the community and now this it is a very cool paper a very cool series of papers and it's very cool


that actually they release the code and they made the code such that it is super duper efficient even like on a single machine and that was very cool because you know being Google they could have just released code that is very efficient on a distributed data center and they didn't do that so that this is it's sort of not really like today anymore or like today when they release code it's always you need you need like 50 cloud TP use to do it and it's still cool that they release code but this was this was really a step


Suat CS 287 fall 2019 sorry to miss you last week you covered on Tuesday contacting variant optimization with Igor and Thursday I believe you covered motion planning with one of the TAS Harry there was a couple of slides that were uncovered at the end of last lecture I'm gonna let you study those on your own essentially there's a topic of lqr trees it's not gonna come in the homework but it's quite interesting it's a way to combine motion planning with lqr to have a more efficient motion planner effectively because motion

Lecture 11 Probability Review, Bayes Filters, Gaussians -- CS287-FA19 Advanced Robotics


planning tends to be very local wind sampling based lqr is a not that local it has a basin of Attraction and so combining both of those into lqr trees is pretty interesting and then the other thing you didn't cover was shortcutting typically when you use a sampling based motion planner you will at the end of a path that's very jagged short cutting is the idea that you just check if any of the steps along the way you can just skip because it's just a little bit of a detour and use a straight line instead of course depends on the dynamics of


your system if your system has peculiar dynamics you cannot just do straight lines but if your system is able to follow any path except when there are obstacles thing you can do straight lines if there's no obstacle so that's shortcutting is a little more detail in the slides and often actually you also run non-linear optimization for control actually at the end to smooth out your path and get a locally optimal path so those are the highlights of what you didn't cover but we're gonna not cover in more detail but the slides have the


detail so they will switch to sometimes the second part of the course first part of the course was all about finding optimal actions how do we given a dynamical system some environment really find a sequence of actions or a policy that optimizes expected reward now we're going to look at the complementary part which is trying to make sense of our sensor readings such that we actually even know what the state of the environment might be and then act against that and we'll need a lot more probability here so we'll start with a


bit probability review then we'll look at Bayes filters which are going to be the core of what we'll be doing and we'll look at base filters today in the simplest setting which is just discrete state discrete observation and then we'll look at gaussians which will allow us next lecture to do it in continuous state spaces and in many ways similar to how lqr give us local solutions similarly with Gaussian siva for nonlinear systems we'll be able to find local approximations to the probability distributions all right


any questions all right then let's get started so why might we care about probability in robotics well often the state of the robot and the state of its environment are unknown and only noisy sensors are available so we don't have a sensor that just says here is the state instead you measure something else probability provides a framework to fuse that sensor information to a reasonable estimate of what the state might be and so the result of what we would compute would be a distribution over possible states that the environment and robot


could be in there's another reason we care about probability which is something we've already covered a little bit which is that the dynamic is often stochastic so we can optimize for a particular outcome but the only optimize obtain a good distribution of our outcomes and again probability provides as a framework to deal with that we've done that in simple settings so far but we'll expand that later and we'll actually bring both of those together in a future lecture where we'll look at the notion that the actions you take could


be taken to reduce uncertainty about the world so you actively go seek out sensor information that could help you understand better what the state of the world is and then collect more reward formulas are called pom the peace partial observable Markov decision processes and that will be somewhat the culmination of bringing together what we're covering in the next few lectures and what we've covered in everything so far let's look at an example how after what would be the state position of the helicopter orientation velocity


angular rate what would be the sensors you have you don't have direct access to position orientation velocity angular rate you have access to some of those in a noisy way and then others you don't have access at all so GPS gives you a noisy estimate of position sometimes also velocity typically only up to a couple meters accuracy so not super precise but it's a noisy estimate of position typically you put the inertial sensing on your robot so for helicopter maybe you have a three axis gyro so charr is a angular rate sensor so it


gives you three three numbers at any given time measuring the angular velocity around the three main axes of the helicopter if that's you know if you mounted axis aligned then three axis accelerometer accelerometers measure acceleration but not exact acceleration it's a little trickier than that Adashi measures all zeros in freefall so if you're free falling an accelerometer measures all zeros and anything that you do that's not free-falling for example if you're standing on the ground your accelerometer will measure 9.81 m/s^2


opposite to essentially gravity because you're resisting gravity and that's what it's measuring at that time so in some sense accelerometer is not not just an in fact much more accelerometer is about measuring orientation very often than it is about measuring your actual acceleration because any resistance of gravity you'll be able to measure and from that get in some sense especially if we're on the ground understand where gravity is pointing which gives you a lot of information about your orientation then three axis magnetometer


which well what is it magnetometer it measures the earth magnetic field so earth magnetic field you might think of it as North but it's actually if you measure it in 3d it's not really north it's roughly north but it actually also points into Earth but it's known which direction at point since you are measuring in the frame of your sensor where is the magnetic field pointing and that gives you information about the orientation of the system that you have some tricky things that of course if you have something that generates its own


magnetic field like maybe have some magnets on your system where you have some high currents that induce magnetic fields then it might perturb those measurements and not really get the earth magnetic field but something else are you flying near a power line or something it might perturb the measurements you're getting dynamics there is noise from the wind there's unball dynamics and engine in the servo motors and in the blades of the helicopter so overall we don't really have access to state directly but we have things that relate pretty


closely to state and we should be able to distill from that a likely estimate of the state or a distribution around some mode maybe of what the state might be how about a mobile robot inside a building the state could be position and heading is let's say a slow robot is just slowly moving you don't care about velocity just the position in the direction it's facing then sense source odometry which is sensing motion of the actuators for example wheel encoders so on your wheels you might measure how much your wheel has turned if you didn't


know the diameter of your wheel you know how much you have moved well that's approximately true because the wheel might have slipped there might have been bumps in the road where it's if it's not flat you don't move as far forward as you thought you would have because you really have been going down and back up but it gives you some kind of measurement of how much you've moved then often people with a laser rangefinder on a mobile robot why is that it sends out a laser beam and then it measures how long it takes for that


beam to come back and that gives you a measurement of twice the distance to the obstacle in that direction and so it's a nice way to directly measure 3d it's a little bit problematic at times because sometimes you have mirrors and then the thing doesn't come back your reflects off in another way or you might have glass that acts much like a mirror and the same thing will happen you don't get the measurements back but if you have nice matte surfaces it's a really nice way to measure how far away things are dynamics is noisy because of wheel


slippage you might say my will turn this much but what if it slipped and you don't know what your next state is going to be if you don't know how much it slipped and it could be unmad unmodeled variations in the floor that effect essentially where you end up all right so those are two examples just to motivate why we might want to deal with probability distributions in that yeah if this is what we're faced with or the helicopters what we're faced with we're now going to have a deterministic notion of this is the state we're just


going to have a bunch of sensor information and the result that best will be a distribution over possible States okay so today what we're going to do we're going to a probability review hopefully that's a review if what we're doing the probability review section is not review you should go study to make sure that it feels like review as soon as possible then we'll do Bayes filters where we will look at the foundation of a lot what we'll cover in the discrete setting and then we'll start looking at gaussians which will form the foundation


for what we'll do a next lecture to do it in continuous space all right so probability theory as a lot of math it starts with some actions and and everything follows from that so probability theory actions here is that you have some outcome a and it could be many possible outcomes an outcome a the probability of the outcome is assigned a number between 0 and 1 maybe you know I don't know the probability of being in the correct room for lecture is some probability assigned to it then the probability of the union of all possible


outcomes Omega is essentially everything that could possibly happen the probability of everything that could possibly happen something in that said to have happened should be 1 otherwise that's not the proper Omega that captures everything and that also means nothing can have a number sign higher than 1 the probabilities don't go above 1 and then probability of the empty set well the empty set is never containing the outcome that happened whatever happened is something it's not in the empty set and so probability of that is


0 and then the main thing here is essentially how you kind of keep track of probability the probability of the union of two possible outcomes a and B is a probability of a plus probability of B but it's possible that a and B happen at the same time and then you're double counting down as you subtract back out the probability that a and B happened at the same time which is a intersection B and that gives you the probability of the Union pictorially looks like this you have your Omega space which has all possible events in it then there is


outcomes that make a true there is outcomes that make be true and then there's outcomes to make a and B through the intersection and the probability of a union B is probability of a plus probability of B minus probability of the intersection one abstract or abstract or concrete depending on your mindset way to think of it is that everything that can possibly happen is a point in this rectangle pictorially that's like every possible state the world could be in is a point in that rectangle I mean we might never know the


exact details of it but every possible state is a point in that rectangle and then a is a property of the state of the world that holds true for all the ones that line that blue circle the bees have property of the state that holds true for all the states of the world in the yellow circle and then a intersection B is a property of the world that holds true for all the points that are in the intersection and so you can think of a and B just as properties or descriptions of you know abstractions of the world of


the state whereas individual points in that rectangle are in some sense the full state of the world which are probably never observed will never really talk about because we don't care about all the details we'll talk about a and B but in terms of how it's all set up that's an easy way to think of it then when you use these actions you can come to new conclusions I mean that that's kind of how math works you posit some actions in the you do some derivations and you have a new thing that's useful to use so for example


probability of a union on Vega minus a well on the Left we just look at what's in the parentheses is well a union with omega minus 8 that's omega so the probability of Omega is 1 that's what we derived up to here on the right-hand side we use that thing that's a probability of Union is promotive each individual event- probability of the intersection that's this we've worked that out then we know that this probability is zero and now we have the probability of a plus probability of the complement of a has to be equal to one and so that's the


kind of properties you would derive all right so there are discrete random variables and continuous random variables so here we have Omega again the rectangle and again think of every point in the rectangle as representing one possible state of the entire world and then X is our random variable we work with X can take on four values if the event in indicates the world is in a state that's in that leftmost rectangle we say x equals x1 if it's in that triangle above it's x equals x2 and so forth and so there's only four values X


can take on the probability associated with X equal X 1 we can think of like essentially the mass associated with that region in the rectangle and we call P a probability mass function a simpler example than the abstract rectangle there would be a coin flip heads and tails probability half for each that's another example of a distribution again you can think of as in the context of that rectangle though you can say I have a rectangle I split it in half there is heads and tails and you know have the states of the entire world my coin came


out and heads have the states of the entire world my cone came out tails we're ignoring everything else about the world we're just looking at heads versus tails and not the details of everything else in the world continuous random variables X takes on a value and some continuum you cannot now associate the probability with X taking on a specific value because if it's infinitely many things it's a continuum if you want to assign finite probability to think so it'll sum up to something higher than 1 which we can have it needs to be


something to 1 so we can so many more rash you have to integrate now so we'll do is this a probability density function is defined saying that if you look at mass under the curve so the probability for X being an interval A to B is the integral from A to B of P of X DX for example if we think from zero here maybe till this point we could take the probability Mouse and that's then the in the area under the curve would be denoting how much parabola we associate with X landing in that interval so again we can't really talk about X stick on a


specific value here the best we can do is say probability of X lying in an interval you could say X probability of X taking on a specific value would say it's zero but then to have something meaningful you take intervals and assign probabilities to the intervals any questions so far all right then the most important thing and what we'll be working on is to look at distributions that involve multiple variables so what why is that well the reason is that has already talked about we'll have things like real state that


we care about and sensory measurements so I want to somehow talk about the joint distribution between the two so if we measure one of them we can say something about the other one or we'll deal with dynamics of the world and so we want to relate the distribution now with the distribution at the next time and so both of these involved joint distributions over two random variables rather than just looking at a single variable all right so the joint distribution over x and y is denoted this way then in simple scenarios


simple scenarios X&Y might be independent meaning of the probability of x and y taking on specific value small X small Y is a probability of X taking on value x and y taking on value of y that's the case actually knowing X does not tell you anything about Y or the other way around so it's actually not so interesting case mathematically simplifies things but yeah if you had a sensor where your sensor is independent of the state of your system the sensor cannot inform you about the system sometimes the property that holds true


though and then it's it's good to know and good to take advantage of mathematically but you don't want to build your systems that way necessarily then X given Y is the probability of X taking on value of small X given Y has value small Y so what we have here is that as a definition we write it as X given Y is the probability of x and y divided by probability of Y so it think again about that rectangle there's a region where Y takes on the values small Y and that has a certain surface and that's py and we look at the region


where X takes on value is small X within that region that's B X comma Y and then the fraction of the surface it takes up that's the conditional we can also rewrite this as the probability of X comma Y is the probability of Y times probability of x given Y this will become very useful many many situations because let's say we know the distribution for Y and want to know something about X which might be the distribution at the next time slice and if we notice in about X given Y this may be the dynamics of the system to go from


Y to the next time X then this equation will tell something about the joint and hence tell us something about X then if x and y are independent then the conditional of X given Y is the same as the marginal for X so P of X will call the Marshal for X and so that means that sometimes in our math we'll be able to simplify things you have an assumption X in planet of Y we see X given Y appear we can just get rid of the Y and simplify to just px all of this is also true for probability densities it's just in densities often it's a small P but


other than that you can write the exact same math and I will say something about the densities instead of the actual probability mass now let's work through the equations we'll be using the most all right so there are essentially two equations there are two equations we'll be using a lot and so let's explicitly step through them and see what they are one s cold law of total probability was it say it says that and we'll write it for the discrete case the probability distribution for X is equal to so probability x equals x is sum over all


values Y can take on of P X comma Y so the reason that will be so important is because often we'll want to get rid of some variables and move to another variable and the way we're going to be doing that is by constructing a Joint Distribution over those two variables somehow and then something out the 1 we don't want anymore more specifically the way this will typically be used is here we'll say well PX is equal to sum over y and imagine we already had access to the distribution for y and we need to construct a joint because you want to


bring in X now we might have a model for how X is related to Y so you'll have px given Y and so this equation here is one that we'll find ourselves using a lot one of the two equations we'll be using over and over same thing can of course be done for densities and densities this summation would become an integral because you have to integrate over all continuous barrels values the variable can take on whereas in some in discrete variables you just sum otherwise same thing though in general whenever there's a summation you can put an integral same


math we'll go through so that's one of the things we'll use a lot the other thing we'll use a lot is Bayes rule Bayes rule hopefully sounds familiar where how do you derive it we actually derive it from looking at this the expression for the joint the X comma Y is px given Y x py which is also because the roles of X and y are arbitrary it doesn't matter which one comes first or second in this equation here we can swap the roles and we have P Y given x times PX from that using just this part over here we can write P X given Y as P y


given x times px over py let's interpret this equation for a moment so what are we doing here we're interested in finding this mission over X given Y so we can associate a story with this imagine X is the state of the system and Y is a sensor reading we might obtain and so we want to know what's the distribution of our possible states of this system given we have a sensory reading Y well it's not easy to come up with a table for that it's not easy to just say oh you know what I'm just gonna build a table for a px given y because it's


actually not really how nature works the way nature works is that if there is a state that state causes the readings so this model here is really the causal model that we have available given a state it will cause a distribution over readings and when you build a sensor and you try to sell to people this is your sensor model that you would provide it said this is the distribution of our outcomes given this situation this is fishing-rod come given this other situation and so forth for example for a laser range file you might say well if


there's a map surface at a certain distance the reading you'll get back is within maybe two centimeters like maybe with denigration 2 centimeters around the proper distance reading something like that so that thing is your calibrated sensor and that you can you can get a distribution for but what actually ugly will be distribution over state given that reading will depend on the prior X what you thought ahead of time might be likely states of your system and so they combined together and then of course is a normalization to make sure things sum


to one that's old is this and often will write this as just 1 over Z times P Y given x times px or it turns out in a probabilistic robotics book notation they love to use it I so often write as a dot x py given x times PX so we don't have too much worry about the thing at the bottom that's just a normalization we just want to understand a thing at the top which brings together the thing for which we are able to build a model which is the relationship from X to Y and how we can then use it assuming we have a prior over X use it to get the


distribution we really want which is distribution over state any questions about this these two equations are essentially the ones we'll just use everywhere and what we'll do today next lecture lecture after that if you fully understand those you should be in really good shape now we can have a new version of this so law of total probability again but with conditioning what does that mean for both of these imagine for the first one here imagine we had already something to conditioned on maybe there was a measurement in the past that was


already there like we really just in not going just from Y to X but there's some Z that's already present somewhere so what can we do well we can say okay P X given Z is equal to we write the same equation sum over Y P Y times P X given Y but since we conditioned on C we need to ever wear a condition on Z and if you're ready I mean notation wise condition on Y and Z is just Y comma Z you don't need to draw another bar and so what we see happen is done the exact same equation can be written with just additional conditioning on Z or any


number of variables as long as you consistently put them everywhere so you can always add more conditioning and so often that's the version we'll be using this will have many things we've already observed in the past we want to conditioned on and then we'll want to apply the equation the left law of total probability and we'll just carry along everything so do our math as if it doesn't exist we just need to consistently carry it along very straightforward got to be careful you can't drop it anywhere same thing is


true with Bayes rule what if you have multiple things you are conditioning on a already condition of something else no problem Bayes rule with conditioning and the reason this should really be a surprise is you go back to the original picture of probability right then you we have this rectangle and the rectangle is really the foundation of everything with a point in the rectangle corresponding to the full world state and then regions correspond to values the random variable takes on if you condition on something


essentially we're just redefining the rectangle as being only only let's say this part that's our new rectangle and that's the only part of the world we're working with possible States you can take on and once we've done that we put all probability on this thing we kind of renormalize it everything's there that's our new rectangle and the maths not going to change because we're just looking at this sub part of the whole rectangle but of course you need to do it consistently every step along the way we need to consistently look at that sub


part where we are having Z this would be Z equals Z we need to keep it consistently to that part we can sometimes have it sometimes not when we do this but then same thing with Bayes rule yeah initially it was for the full rectangle now we restricted attention to some sub region no problem we just condition everywhere P X given Y comma Z is equal to let's it's equal to well what do we have we had Y given X but of course also Z because Z needs to be in the back everywhere now then P X but we need to carry along the Z and then normalize by


Y again given Z but again the bottom here doesn't really matter much in baseball we usually know that we just need to compute the top part for every possible value of x and once we computed the top part for every value of x we can see what it sums to and whatever it sums to is what goes in the bottom right and so applying Bayes rule is about doing this for all X and then summing to know what the bottom is and so here we see that we can incorporate new evidence we already had evidence we had already had X given Z we had a past observation


distribution over x given that observation then a new thing came in for Y and we have a model of how Y acts given X and Z and practice often the Z will disappear here because Y will only depend on X not on Z but in channel YK depend on X and Z and then we can combine that together to know what now new distribution is over state X then another concept that will be very important is conditional independence so conditional independence is a lot like independence but a little less strict independence is a property done and sometimes it's nice


because it simplifies the math whenever you run into it but if your variables are all independent then you can't really infer anything from one variable about the other variable and so ultimately you get no interesting results out you need the variables to interact to get something interesting out now it could be that they interact but in limited ways and that's what conditional independence tries to capture so condition dependence we have X conditionally independent of Y given Z if and only if we have P X comma Y given


Z equals P X given Z times P Y given Z so it's like independence the joint between x and y is the product of marginal for X marginal for y exit that it's only true when you condition on Z once you know z x and y are independent but as long as you don't know z they might have a relationship for example z might be something I don't know like Z could be the weather and X might be is somebody carrying an umbrella and Y might be is the is the pavement wet or something and once you know it's raining you you don't need to you already know


that the rain will cause x and y independently it's not related directly whether pavement is wet and somebody cares umbrella it's through the fact that it rains which is the common cause for both of them and said I'll make x and y independent given see another example and the one that we'll see most often in this class is if you know the state of the world then every sensor for every sense you're reading might be independent very often because if your sensor just acts on the real state of the world then the reading that happens


first very happens next will be unrelated and we'll be able to simplify things okay so this is this is kind of really everything we need so we just need three things we need law of total probability the regular version and the conditioned version we need Bayes rule the regular version the conditioned version and we need a notion of conditional independence which will allow us to simplify equations this is also equivalent to writing and also we'll see it that way equivalent to px given Z and Y but once you know Z X is independent of Y being


equal to px given Z and it's also equivalent to P Y given Z comma X being equal to Y given Z this once you know Z X does not tell you anything about Y now to be fair this condition the dependence assumptions that will be making sometimes in the real world might not exactly be true ever but they'll often be reasonable approximations and we might be willing to make him because they drastically simplify the math and the competition we need to do to then run the algorithm in practice so often often these things are assumptions we


make to approximate things that are almost independent in the real world questions so far yes mm-hmm so typically the way you would do it is this typically what you'd be given is from a previous calculation maybe or just given would be this thing here which is the prior over X of PI over X given Z you'd be given a model of how X and Z in this case are just X in that case cos Y that's your sensor calibration model for example and that together allows you to compute this product for every possible value of X because we're conditioning on Z a


favorite possible value of X yeah we're computers for every possible value of X because Y will be measured in this scenario Z will be measured so y and z will be numbers available to us x will not be known can take on many values and so we'll compute this for every value of x now after we've computed this for every value of x we'll have a table of all these four values of X and those entries if we ignore the bottom will not sum to one but we know it's supposed to be a distribution over x given y and z so then we know I also know we we


ignored the thing at the bottom that does not depend on X that's key here there's no X in here so what we're computing is a table with entries that depend on X every X will have its own entry and we know all of them are missing the same rescaling which we've ignored and we but we can find the rescaling by remembering that actually every all the entries in the table have to sum to one and so summing all the entries in that table will actually compute this thing over here mathematically we know that's a way to


compute this and practice we don't often pay attention to what it really means we just say hey we want this we know this thing here does not depend on X so we can ignore it as long as at the end we do a rescaling of all entries with the same scaling you cannot rescale one entry for x1 by the other one the other you need a fixed rescaling across all entries that you computed so the things written on the board we also have on the slides where often we have a pairing between the discrete version where there's a summation and


continuous version where it just replaced out some sign with a little squiggly thing and then you put a D something is over the variable instead of putting it under so essentially just always a place sum over the variable with this thing here and then a dy and then you got the same thing just for continuous space Bayes rule the terms we didn't name them but essentially often the X the P of X is called the prior because we have it ahead of time Y given X is the likelihood of a measurement Y given this state is x is often what it


ends up being solvent called likelihood and evidence is yeah that's just Y is what you measure to is a probability of getting that evidence this we just talked about when you do these calculations often you effectively algorithmically you compute it for all X's you compute that top part in the fraction and after you've done that you know it has to be rescaling you just compute the rescaling and multiply with that in the paralytic robotics book notation it's a little bit funny that sometimes they use the ADA as the thing


you multiply with sometimes I think you divide by just gonna I guess be it be okay with that it has different meanings in different contexts but ultimately it's just a rescaling and some people like to divide by the risk and some people like to multiply with one over that thing all the same law of total probability with integrals we saw that basil with conditioning we saw that we saw conditional independence what it means again it's an assumption we often make that will hopefully be approximately true or maybe even fully true so we can


use it to simplify our math here's a simple example so we have a robot trying to see if the doors open or not okay what's the probability of doors open given some measurement of the sensor of the robot it is to illustrate that kind of reasoning behind why we often use Bayes rule probability of being open given some measurement is a diagnostic distribution it's kind we're trying to diagnose what's the state given a measurement but those distributions often we cannot build we cannot deliver a sensor with that distribution because I mean you can't do


that imagine you deliver the sensor to somebody and they lock the door and nobody has the key anymore then your sensor can't know that because you didn't know that ahead of time what you really can build is the opposite distribution that causal one given a situation what am I going to measure and so that's what's be available to us and that's what we have to use but it's not what we want the reason about the world so we'll use Bayes rule to get what we want which is the opposite thing going from the distribution that we know how to model


which is the measurement given the state to the distribution we want to take action which is one know whether the door is open or not or at least a distribution over that for the causal one it's really just a matter of counting frequencies you might have a physical model that makes it easy but often you just kind of just do a bunch of measurements so you see how often does a sense to read this or that as a function of whether the door is open or not and now I have my model let's do a concrete example we have we need to put


some numbers into it probability of a measurement Z given the doors open is 0.6 Zija the door is not open is 0.3 so we see that when the door is open we are more likely to measure Z as the outcome than when the door is not open so if we measure Z which should increase our probability that the door is open and our prior is 50% well we can apply Bayes rule we measured Z we can look at Bayes rule fill this in do the math which is described to find indeed that now after measuring Z which favors the door being open we have more specifically 0.67


probability that the door is open now you could have another measurement imagine of another sensor wait wait go yeah let matching we have another sensor z2 how can we integrate that well we can again apply Bayes rule we already know that we can apply Bayes rule again even with conditioning in the back we already conditioned on one measurement just in a condition on the next measurement we actually do this as many times as we want so the condition on many measurements we can do it one at a time and build it up so here we have the


often a assumption is applied to simplify the math is that the sensor measurements are all independent given the state of the world because the state of the world is causing what you measure and so if the state of the world is a known entity then each outcome is independent given that state that will simplify the math because then when we look at Z M the enth sensory measurement given the state of the world X and all the other sensory measurements we don't need to build a model for Z and given X and the other measurements we just need


to build a model for Z and given X the state of the world which is exactly the kind of sensory model that we like to build so we get a simplification happening over here we had ZN conditioned on everything that's the standard Bayes rule with conditioning but then we made an assumption there's not generality but we made an assumption that Z and is independent of the other Z's if we already know the state X and then this here has simplified then again the bottom we don't worry about there's no X in it we can compute the top part


for every value of x once we're done we sum it all together to know what the bottom is that will normalize it we can apply this repeatedly that the idea we just saw that this becomes Z on given X px given all the previous ones there's nothing specific about Z N and we'll see then we get a product over all Z is given x times prior over X and so now all we need is that sensor model for each sensor given X we multiply probabilities together and we get our posterior distribution okay for the door example we had in this


case a new sensor is z2 and when we see z2 it actually is more likely to be seen when the door is not open we observe z2 so what do you expect to happen mathematically we had a 2/3 probability of the door being open ahead of time after sensor measurement 1 we observe z2 that probability should drop but let's check we can do the math we just apply Bayes rule fill in the numbers and indeed the probability drops as we expected all right now there is one thing I want to point out here is that a typical pitfall is that real-world


sensors are not always super independent so you might have a sensor and you might say okay maybe I have a laser rangefinder and it reads a reading but then maybe send out two beams next to each other but those two beings might encounter the same kind of noise or the same kind of interference in their paths and so those two measurements might not be independent and the assumption we just went through where there's all simplifies just becomes a product of things is violated even though often in the math will want this and practice be


careful you need to think carefully are these sensory readings really independent or imagine the laser rangefinders reading something at time T and you run it again at time T plus 1 but nothing has changed in the world then are those really independent readings not totally necessarily because you're measuring the exact same configuration with the exact same sensors so maybe it's not completely independent so be careful what can the effect be imagine you imagine you think they're independent these readings and they're not what do


you think will happen let's think about this imagine we have really one reading and really what we should be having in this equation because there's only one in one measurement happening all the others are just copies that are not independent information they're just copies of that one reading that's how dependent they are extreme case of dependence we're just you know instead of fitting it in once into our Bayes rule thing we say hey why don't we feed it in ten times because that way we can incorporate the information even more all right what


would happen you feed it in ten times you multiply this thing in ten times so what will happen is that whatever that measurement is favoring you'll become overconfident in it if your favors door open and maybe normally with one time incorporating that measurement you go from 1/2 to 2/3 if you cooperated a hundred times you multiply it in hundred times all of a sudden you'll be at 99% probability instead of where you should be which is 2/3 that's exactly what happens if we look at the graph here so here is a graph showing as a number on


the horizontal axis number of times the same measurement is incorporated using the math that is the assumption that these measurements are independent and we see that very quickly you can actually flip the probability from being 99% being one observation one state in the world to the other state in the world X 1 versus X 2 so be very careful about that another places might come is like you might maybe have I don't know accelerometer or gyro and the gyro is supposed to measure independently the angular rate of your system but the gyro


is a physical system under the hood or something physical happening there and so it might take time for that physical thing to settle in and really get that measurement and if you you know read it off I don't know ten thousand times per second that might not be valid that might only be 10 or 100 independent readings really and if you then use the 10,000 readings instead of just the ones that are actually independent you'll become wildly overconfident in whatever it is that you're measuring rather than keeping a reasonable


distribution of our possible states of the world okay let's take a couple minute break here and then let's look at Bayes filters and gaussians actually much more I did it's not as unstable for alright let's restart any questions about the first half yes so I've in a lot of things we assume a first-order Markovian system and kind of touching on the example that you were talking about at the end have you seen in practice whether maybe the first-order Markovian assumption isn't appropriate but a certain like n step Markovian assumption is appropriate


so that we can find this balance between having something that we can get accurate I guess quantification from but also the joint distributions are not intractable yeah so it's always a trade-off between tractability and match with the real world I would say I think in general first order assumption I mean in principle if you have full state then of course definition of full state means that the first-order assumption will be true otherwise it's not full state but in practice your state will not be maybe the full state of the world to be a


approximation of the state that you use as your representation for state space and as a consequence first sort of assumptions might not exactly be true I think often it's dealt with in pretty much ad hoc way where people just look at the system and say ok for this system it seems like we need to look this far back or that far back to get what is effectively the state of the system by including enough history but I haven't seen like if necessarily like a very clear-cut way to define it in general and this ties directly on what we're


going to do now which is Bayes filters so often the world is dynamic because robot does things other agents do things time is passing by the world changes in various ways as a consequence so how to incorporate actions into what we're doing as well as our sensor readings so typical actions robot turns its wheels to move robot uses manipulator to grasp an object a plant grows over time and takes up more space actions are never carried out with absolute certainty so in contrast to measurements actually actions generally increase the


uncertainty dibal you have is you will have measurements reduce yesterday in your distribution then action gets taken which introduces new uncertainty and the distribution becomes higher variance how to model actions well we can again do it with a probability distribution so we can say the prohibition of our next state X prime given current state X and action U and that again is a causal model it's something we can build a model for and then use to calculate the distribution over next States so for example a robot might try to open a door


and it might succeed or might not succeed you might have a model for that so you might say if the doors already open and or in this case closed the robot tries to close the door doors already open well 10% chance it'll just stay open because it fails 90% chance it managed to close it if it's already closed and it will keep it closed okay that's a model that might or might not be correct but these are the kind of models that we're going to be working with such that we can do calculations about possible states of the world based


on our actions we took and sensory readings we get well this is exactly the law of total probability we're applying here the conditional distribution for X prime next day given the action U is well we have a distribution for current state that's what we assume we assume that we know the solution for current state will then multiply in X prime given U and X to get to joint over X Prime and X once if they joined over X Prime and X we actually just want X Prime so we sum out or integrate out X to just get the thing


for X Prime that's exactly what's happening over here loft all the probability with some conditioning so we can run this for the robot example and I'm going to step through the details here but you can do you know with those numbers we have the math you can see okay given the action that we took the probability of clothes become 15 out of 16 and the priority of open is 1 out of 16 how about measurements measurements are what we've been talking about the most so far we'll use Bayes rule Y again because we have easy time getting a


model for sensitive reading given State but we don't know how to get a model for state given sensory reading directly because we need to know the prior over States and wanted to use Bayes rule to get that now in Bayes filters the framework is that we get a stream of actions and observations we have a sensory model a action model or a dynamics model and a prior distribution for where the state of the system is starting from so these are our Givens what we want is an estimate of the state of the system at all times so we want to


keep a running estimate over distribution over possible States for each time T pictorially often it's drawn this way this graph captures the causal relationships that are present in our set of assumptions so X again is the state over time use the actions we take actions will affect the next state the previous state will affect the next state and the observation in this case assumed to just depend on the state the state of the world determines what we might measure and so when we get to look at this graph the explicit assumptions


being made is that the measurement at time T given everything that happened before everything happened before only depends on X T so for no X T no past states matter when we know X denote past measurements matter no past control inputs matter it's all summarized in state XT we're going back to the question of course if your state space doesn't capture everything about the world then this will not be true in this assumption will be violated and typically that's the case you never captured the whole world stayed in what


you do but hopefully you're close enough that this actually works then same thing for dynamics next state could in principle depend on everything that came before but the assumption we make is that it only depends on previous state and action taken again the paralytic robotics book makes some kind of different kind of indexing that I'm personally used to but since that's the the best book to go read probably on this if you want to read more I'm sticking with their annotation so they said the action taken UT is the one you


take that will land you in XT normally you would call that UT minus 1 I think that's what almost everybody calls it but in the book they call a UT it's the action that you know you take right before you land in state X T so that's what it is and that's also reflected in the graph over there UT it's actually you take right before you end up in state X D so unlined assumptions are independent noise meaning that the this thing here these two assumptions means that if you let's say how did the mystic model plus noise that the plus noise


effect is independent at different times and it also assumes if you are if you're going to say whatever I get out of these calculations is the correct distribution that's only true if these assumptions are correct if you make any approximations in your dynamics model in your sensory model well those approximations will result in miss estimates that you end up with but it's the best you can do but keep that in mind it's there's a lot of assumptions made here and so you only get the result relative to the quality of the


assumptions that you made the quality of the models that you built in okay let's step through Bayes filters so what builds up step-by-step is the probably the key thing for the second part of lecture so we want to believe which is a distribution over X D given everything from the past including also the current observation of time T now remember it's going to be a recursive thing we're going to already have this presumably for time T minus one and we're going to see if we already had it for t minus one how do we get it for time T what's the


latest thing that happened latest thing that happened in that progression is the measurement ZT how do you incorporate a measurement Bayes rule so ignore everything but XD and ZT what we do p XD given ZT we'd say Bayes rule its ZT given XT times XT okay that's exactly what we're doing we're going to apply Bayes rule and then some normalization right but remember there's also that other stuff and we'll just carry it around everywhere we know that we can carry things around everywhere as long as we do it consistently so it's


just Bayes rule apply now we had some assumptions that the measurement at time T given XT does not depend on anything else so we just simplify accordingly okay so we've not yet gone from XT minus 1 to XT we've incorporated the last step the measurement can we now do the transition from XT minus 1 to XT well let's think about it how do we go from XT minus 1 to XT well xt- one I've already have it we're multiplying the dynamics model to get the joint and then sum out XT minus one okay that's exactly what we're going to do we


now say the distribution for XT is XT given XT minus 1 times distribution for XT minus 1 and integrators sum out XT minus 1 and sure we're carrying around all this extra stuff here you want or UT and the Z's and so forth but we know that's just extra stuff we can care around consistently this is just the law of total probability for XT and XT minus 1 then we say well can we simplify this because our mark of assumptions yes we can we don't need to conditioned on everything that's conditioned on there XT given everything in the past only


depends on UT and XT minus 1 and this here is now our complete recursion because this is a thing we assume we already have is the same thing we wanted to find for XD we have it for XT minus 1 and now we can just repeat all the way back to X 0 and we're good to go so at every time step as we track our system we can incorporate the controls we applied U at time T here gives the tribution over XT this thing over here gives distribution over XT given that what we already knew about XT minus 1 and the controls we applied and then we


get a measurement we're multiplying the likelihood of the measurement and then we were renormalize and if our distribution for XT and we can keep tracking over time all right the mark of the assumption makes this even slightly simpler notation and we're good to go all right so and then the last thing here is just the notation the book uses BL belief of XT minus 1 is a shorthand where there's no there's no assumption here it's not like we can delete all this stuff here just to be clear this stuff is now being


deleted by some assumption is just the notation beol XT minus 1 by definition means XT minus 1 given all this is just a shorthand notation so for the second-to-last equation or like the third line from the bottom and the second line it goes UT at the last one and then it's CT minus one what happened there like on the conditioning oh ok the only thing that happened there essentially that the UT was removed yeah it's not fully spelled out but essentially UT does not play a role for XT minus 1 because it comes after and so


UT was removed from that that's all that happened and so this is the final result we have we can put this in an algorithm and this the algorithm that will be running and this is the version for discrete States of course where we can just track the Blee compute this was actually very simple you start out with some distribution prior over X your distribution over X then when you get an observation you're multiplying the likelihood of the observation given X of course if you do that for all possible values of X you sum that together


normalized by it you every new believe for X after that observation has come in if it's not an observation that came in but you took an action well then you use law of total probability then a much smaller gets multiplied in you get the next state and so you can run they don't if you alternate between Z and you you can just have a bunch of Z's coming in incorporate them then take up some actions incorporate that and just repeat over time whenever something come in act upon what happened so in summary Bayes rule allows us to


compute probabilities are hard to assess otherwise that's really what's going on here under the hood it's that we don't have direct access to the distribution we want we've access to it sometimes the reverse distribution we can use to get what we want under the markets assumption this recursive Bayesian updating can be done very efficiently and Bayes photos are kind of the common pricing tool for estimating the state of dynamic systems here's a simple example imagine your robots supposed to localize has four


measurements and that means measures in all four directions and it checks whether it's a wall or not in each of those directions it's a noisy sensor and we're going to not go through the math here we're going to qualitative let's see what happens well if it's if it's really over there but we of course don't know where it is the gray e the uniform gray means our distribution is uniform over all possible locations we show the location of the robot just to you know know what's going on but the robot doesn't know that robot just could be


anywhere get some measurement to come in since wow I only see things above and below me not to the sides so the Bayes rule update will result in the new distribution then they might say I'm going to move but it might have noisy motion so it then kind of spreads out the distribution a little bit then it measures again which shrinks it again and this process repeats over time and over time it might localize itself in this building of course this assumes that access to a map of the building otherwise this whole


base update rule wouldn't work out in this case okay that's it for Bayes filters I want to now move on to Gaussian so we'll look at univariate gaussians multivariate gaussians we'll look at law of total probability for gaussians which already covered for discrete case we'll look at what does the math look like for Gaussian so we'll do that next lecture but that's on the menu for us for gaussians and we'll look at conditioning Bayes rule how it shapes up for gaussians univariate gaussians what did it look like is this kind of


blob kind of distribution where most of the mass is in the middle but it can actually go all the way to infinity on both sides there's something called standard deviation Sigma and 68% of the mass lies within one standard deviation of the mean and then it kind of decays out from there the mean is usually denoted by mu sanitation by Sigma and then the variance is Sigma squared the density itself is this thing over here so it's exponentiated negative x minus mu square so let's think about that when X is close to MU and the X is equal to


MU that's zero and you have e to the power power zero which will give you one and then if you are the further you go away actually could be even let's see yeah each of our zero would give you one if you go further away from you you start moving away from you this thing will become and the exponent will become a more and more negative number and that will make the expo that negative number will be a lower number and density drops it's exponential Exponential's drop very quickly that so why this nicely integrates to one


still because Exponential's drop quickly enough there's not a lot of probability mass out in the in the faraway parts the normalization constant upfront that's just you know you do some calculus and you find out that to make sure that this quantity here to make sure it integrates the one you need to put upfront 1 over Sigma of 1 over square root 2 pi ok so what are some properties of gaussians well this is integrate to 1 so we know that this thing integrates to 1 the expected value of your variable X is expected value is you integrate over all


possible X can take times probability of taking that value if you work out the integral and not proving that this is the integral here I'm just saying if you work out this integral you'll see the result is mu so the expected value under a Gaussian is immune not to an expected either because I mean in the name is also saying it's the mean but you can actually do the math and principle to find out that mu is indeed the expected value under a Gaussian distribution the variance is defined as X minus mu squared on average what's your square deviation


away from the mean you can do the math compute this integral you'll see Sigma squared is the variance which is what we already called it but in principle you know you have to calculate it verify that the name we use for Sigma squared is actually really what variance means and indeed it is now you might say why might we care about gaussians these integrals I mean we figured them out or somebody figured it about but they're not necessarily easiest but at least they can be done well there's another reason to care about them aside from


them being convenient to work with the central limit theorem classical CLT says that let x1 x2 be an infinite sequence of independent random variables with an expected values mu and variance Sigma squared then we define a new variable ZM Zn is the sum of all of them minus the sum of the averages so this should be centered around 0 because the sum of all the mind is the sum of the averages or expectations and then normalize by this quantity over here it's a scale down then for the limit of n going to infinity with


infinitely many of those variables being put together in a sum and send it around zero and normalized then we have Z going to a zero one so zero mean Santa Aviation one Gaussian so what that means is that if whatever you care about is the effect of multiple independent factors the resultant variable is often distributed more or less like a Gaussian and the more independent factors contribute to the variable you only care about the clothes that variable we ultimately care about will be distributed according to a Gaussian so


it's not just about convenience of math though that's a big part of it another big factor why gaussians are a reasonable thing to use is that in fact even if there's enough underlying factors it might really be a Gaussian the thing you're looking at about multivariate gaussians well here's what it looks like these are densities where X is now a vector so we again see a normalization happening up front which is has a determinant of Sigma in it Sigma is a symmetric matrix the covariance matrix and have X minus mu X


minus mu so whenever you see something like this often it's a little hard to make sense of it directly if you haven't seen it before your simplest thing to you say okay yeah there is matrices and vectors but whatever is just scalar if it's just scale it looks like a single univariate gaussians again you just have X minus mu squared if it's not a scalar what do we have well Sigma is a symmetric matrix so the inverse of Sigma is also symmetric matrix so the symmetric matrix symmetric matrices are as we know from we've covered lqr just


the rotation away from being diagonal matrices just a coordinate transformation away from being diagonal so we can just as well think of it as diagonal matrices if we were just working the correct coordinate system okay so imagine it's diagonal then we really see like x1 and mu1 attracting with the first entry in that diagonal and x2 mutant with the second entry and completely independent no interaction so x1 and x2 are their own gaussians in that coordinate system where now they might have different variances because the


dialogue could have different entries for x1 and x2 but Elmi is just saying oh we have to gosh is 1 4 X 1 1 4 x2 they could have different variances that's it now if we're not on the coordinate line system shirt then it won't look like a diagonal but the intuition remains the same there exists a coordinate system where this would be diagonal and easy to think about I also remember back when we did lqr I told you any matrix can be any symmetric matrix in a quadratic form is fully general because we have a non symmetric


matrix the non symmetric part cancels out anyway now we did some math on the board and that feel free to revisit there's no reason to ever consider a non symmetric matrix Sigma because the non symmetric part will just cancel out in that quadratic form alright so we can compute expectations here to expected value of variable X is of course is mu expected deviation of X from the mean is actually entries in Sigma so this is saying expected value of x I minus mu I so how much bigger is the Exide than its mean x how much bigger is xj then its


mean well if they're both together bigger and together smaller then this will always be a positive quantity and will have a positive number come out if one of them tends to be bigger when the other one is smaller and the other way around then they'll have opposite signs a negative entry will come out and they're completely independent and their bigger is smaller then I'll cancel it be 0 so Sigma IJ is positive when they both together tend to be above average negative when they have counter correlation in terms of above below


average and 0 when there's no relationship between when they're above or below average let's look at some examples what this looks like so here is a plot of a density of a Gaussian mean at 1 0 the first axis is this 1 the second axis is this 1 in all these plots Center deviation 1 for each coordinate so symmetric Gaussian which keeps television while we shift it now the mean is at negative 0.5 for the first query we see it shifted what if we shifted even more will so shifted for the second coordinate to mean again this


bump just moves around let's look at some more examples again a starting point a 0 centered variance 1 Gaussian we then reduce entries on Sigma so Sigma is diagonal here setting stay acts as a line nicely symmetric point six point six it becomes a taller peak because all the density has to be closer because you're not allowed to be as far away from the mean as often so the mouse has to be more centered you can make the standard mission larger and things will spread out you can also do things that are not diagonal matrices so here again


on the left the standard one in the middle we said x1 and x2 are positively correlated so what do we expect we expect that there is a lot of mass along the axis where x1 and x2 are both above the mean or both below the mean the mean here is zero so along the main diagonal we expect there to be a lot of mass which is exactly what we see then here we made it even more 0.8 and we'll get even more mass along that main diagonal because whenever one is above average the other one should also be above average below average the other one


should be below average most of the time you can draw this all so differently so drawing in this kind of 3d like sketches is sometimes hard to do much easier to draw contours so the contours of the densities shown below the kind of 3d plot then here's a corresponding density contour for the middle one and for the last one you see indeed the ellipse runs along that main diagonal how about some other examples here is negative correlation so here it runs along the opposite axis when x1 is above average x2 is below average and so we get it to


run the opposite way we can make it even more negative negative point-eight even stronger negative correlation or we can make it positive again and also make one have higher variance than the other one so the first coordinate has more variance than the other one any questions about these okay then next lecture we'll do the math for the total probability law and Bayes rule for gaussians actually one quick extra announcement for today's today's lecture is probably one lecture where the pace is probably the most off for


so yeah it's a pleasure to be here and today we'll be cooking through the labels a couple of words about me I'm a kegel competition Smith master located in Minsk Belarus my name is Kegel is V dot e dot s dot and currently I work as a data scientist at h21 so today we'll be talking about some distinctions between labeled and unlabeled data then we will talk about what is said at Lackland actually is some use cases and some recipes of how to cook so the label to suit the labels and some example on the real okaycome

How to cook pseudo-labels | by Yauhen Babakhin | Kaggle Days Dubai | Kaggle


competitions where to the labels were applied and achieved good results so here is the general supervisor on problem train and test data and for train data we are given a label so have some kind of wave of data and our goal is to build a model to predict the test labels so test is kind of unlabeled data and generally it is a kind of usual keiko competition scheme where we are given some kind of label data and you need to make a model to predict what the unlabeled [Music] and the problem we'll be talking about


today is the kind of situation when a lot of unlabeled data and label data is very small so this could happen to both in Europe some kind of usual motion journal projects and as well as on Kegel competitions and here probably you could some kind of create some tricky models to build them on this small label data and apply some different techniques but probably the better approach is to somehow use this huge unlabeled data and the reasons why like what will have such situations when the label data is very small the first one is it is expensive


so to label your data you need to acquire some special people or you need to acquire domain experts or use some special software in order to label your data and obtain label data so consequently it is also time consuming so you need for example one month to label one more potential for your data and of course your management will not be satisfied with with this approach that you need - too much time and you need to put the model right away there are some other reasons for example seconds it could be some sophisticated


experiments you need for example build some I don't know very very hard experimented start to establish and there are lots of lots of stress this is a need and it is hard to repeat it frequently and that is why it is also could be expensive a time consuming so it's kind of basic reasons why why we have such deviations with a small label data and here is a quote by entry in there it is not who has the best algorithms that wins it has the most data and probably it is much more crucial nowadays when there are lots of


machine learning models that work in out of box so it would apply and you could solve almost any problem type are just using some some some predefined models and the problem is that you can't apply models when you don't don't have data or this is the reliable data is too small so probably it could all it could also be extended for personal data so if I have small label data's and it's also hard to do some kind of huge supervisors to supervise or model and in case if you're unable to get more label data and if it is impossible or it is hard to


acquire to reduce the message codes same supervised chlorine is introduced in a simple example here on the left you have a classic supervised subra specification a problem where we have for example two classes triangles and squares and our goal is to build a classifier so a decision boundary that would distinguish these two classes so here on the image pad on the image B we have some kind of sample decision boundary between these two classes what same service were allowed to do it allows you to utilize all cells and label data so this is red


dots some kind of way observe this data but we don't we don't have the real labels and actually from this red dots with some kind of get the structure of the data and it allows to sometimes change our decision boundary itself uses this knowledge and we see that this is and right now is more more reliable and more generalizable probably yeah and what a sexual thrill delivers so the labeling it's kind of the simplest semi-supervised learning so simple superest Waldron have has lots of different approaches but see the


labeling is like the most petition and so it's the most easy to use and the idea is pretty straightforward so we have the label data we train Air model like some supermodel one when this label data and afterwards we just make the prediction spawns and label data and actually these predictions are already through the labels so we treat all the test observations is that we have predicted by our model as to the labels and then we could some kind of concatenate these two data sets so initial data and our predictions made by


our models and treat all of this data as a kind of extended version of our label data and users to the labels in our subsequent training yeah before before speaking of how to with how could we utilize to the labels I'll talk about some kind of couple of ingredients the first one is confidence so instead of taking all the predictions from the code test set we're interested only in the content predictions the reason for that is if we add to power of two the labels some observations that are hard to predict or some special cases some


corner cases then it would like been late when been detect our our subsequent training because it introduces some noise and some bias in now in our model and we need which is only the confidence intervals in order to like I select only the reservations that our model is confident in there are different different definitions of what is obscure or what is a confidence prediction for different types of of the problems so for example falsification problem the easiest way is to kind of take the probabilities for each process that were


predicted and for example if at least one one plus have probability over 0.9 then okay it is reliable reliable observation we could edit in the content through the labels for image segmentation problems for example we could use some kind of percentage of motivated or pixels away we'll just find in 110 pixels on the image and then we're treating like the person of course two pixels for example over eighty percent those senses alteration is confident and of course aggregation type of process it's a little bit tricky


because it's hard to tell what is kind of confident prediction for the regression problem but one approach what you could use for example in different problems is that you can look at your predictions from one epic to another unit year training of the neural network and if you see like huge jumps for one equal to another during the nest neural network training it means that probably this observation is unreliable and it is not not a good idea to include it in the to the labels and overall pseudo labels is kind of the


message that is widely used in the in the deployment context as wrist posture is that no metals allows to treacherous terrain online and you can add some new data to either intentional training so but probably not probably so for sure it could also be used for some classic machine learning problems but it is not not so popular so my talk will be more about about some neural networks and the deployment context yeah ii ii degraded in isn't semblance so instead of creating one one model and predicting what one set of students who


could train multiple models sending samples Emmet's in some way and obtain a new set of to the labels the reason for using an semblance observed two different reasons the first reason is that worse it is better to use in similar photos instead of one model if you talking about the quality so single model is almost always be the first versus it's an example of model and the second reason for that is that in several photos allows to get diversity to up to the labels so we for example using only single model from one page to


another and Catina training with a single model sinu put some kind of propagate the errors or homes on this model and if you're using example since the diverse models could just somehow eliminate this effect and we obtain a more generalizable to the labels all right so the first recipe how how we could utilize to the labels about change multi nia sleaze so it consists of two steps the first step is that we just Union two datasets label the data and see the labels obtained and treat it as a mutable data so of course he could from


to deliver secret select one with continent predictions were some kind of only subsets Anderson dish but which is also the labels as well as a column as a label and afterwards this new label data set could be used to train a new model so now instead of using original we use both pseudo inverse sine theta and trainer model and it occurs is it such approach allows to get better results and change with a single model I mean the model is a zero strain on unlabeled data so identity so what is the process it is to concatenate train data and to


the labels all right another approach is called pre-trained it's a kind of a little different in the previous approach was based on some kind of on a data level so we can created our data in the city in the interested single set of labels now we are solving some kind of a model level problem so we take on the schudle labels change the model only opens up to the labels and we have change some kind of weight in civilization so we will save the way so this one is a bit we have obtained and this weights could be


used as a starting point for subsequent train on the label data on the initial table data and as the reason why it is working is that afterwards after you obtain your model or some pseudo labels now you wait wait no information about you your data set about your domain you're working with and this installation works better than for example if a man in image net installation so the pipeline works in the following way we have our pertained models of Australian condoms with to the labels we initialize the weights and it


allows to train faster and obtain better results if a fan changes this model or going to label data on the initial label data and after we look ups to point to you is this model at NU model you could make predictions on general data and again this approach allows to get better results compared if we have started for from here and just insulin insulin ensures the weight with the image net okay so each recipe has some herbs and spices and here we'll talk about the validation when we are talking about pseudo labels case absolutely labels is


kind of agreeing a great way to - or that kind of over senior model and yoke and abused or fits into the leaderboard or fit into a validation data and you need to establish the proper way how to compare the models before you can apply to the labels and after the second version of will be applied and for example have this for four volts away using the basically k-fold cross-validation and the first approach could be we just train four different models for each of the poles and samples them and obtain through the labels and


since these ends the specific labels are used in the first or second recipe in order to continue the training however this approach is weak because we now see the label dataset contains the information about also also targeted target Morocco's also late labels is a train data and if for example after work limited to the labels we want to measure the quality ones there for example for sport then it may acure that our our quality is too too too too optimistic because so the label school reticle since the right labels also causes this


of this first hole so the better approach is to use some kind of out of focus to the to the labels so for each fault the train has separate a separate model and predict and create a separate suitable data change independently and afterwards it kind of provides a reliable reliable schema validation so then we can compare the models before the first ago that was Jack have been applied the only drawback here is that we need to train for different models for each mode and obtain only one set of CB labels and they're not really reliable


because we need this kind of an example ingredient when we are developing multiple diverse models and actually in practice everyone uses these schemes just like a skin so it is it is letter Western model I focus as I said in the kit but the reason for the usage of this scheme is that we are already obtained an assembler model so they don't have four models and if for example for each move with well-developed three different architectures narrow Network center cap over 8 to 12 models and it is really a great world chain of two the labels and