To start off with maximum likelihood, we first need to think about our parameter theta. So this is a parameter that's unknown and we're trying to figure stuff out about it. So even though theta is unknown, what we could do is go out into the world and collect some data. So we would collect x1 through xn, so n measurements. And if they're independent and identically distributed, then we'd have a random sample. So we have x1 through xn, that's a random sample of size n from our PMF or PDF, f of x given theta. All right, so now we've collected our data. We need to think of that data as constant. There's no more variability anymore. Now we're in math stat world where our data are collected and they're constant. So we have this independent identically distributed set of data. And now we're trying to figure out what is a good value for us to guess that theta would be. A good answer would be, well, what is the most likely value that theta could be given the data that we have observed? So we need to find something called a likelihood function. And this is just a function of our parameter given the data that we've observed. So we're holding constant x1 through xn. And now we're seeing if we plug in different values of theta, what likelihood do we get out? So maybe we could plot it and our likelihood function might look something like that. So if we had a value of theta like here, then our likelihood would be this. So maybe that looks pretty high in comparison to this value of theta, which gives a pretty low likelihood. So the higher the likelihood is, the more likely that parameter value is given the data that we have observed. All right. So now we have an idea of what that likelihood function is. How do we actually get that likelihood function? So our likelihood function is just the joint PMF or PDF of our random sample. So we have x1 through xn given theta. But we know that they're independent so we can split this up into the product of univariate. Okay. So, most of the time we can just write it like this. So it's just the product of each of these univariate PMF or PDFs. Okay. So again, we in math stat world are thinking of the data as fixed and we're looking at different very different values of theta. But to get our joint PMF or PDF, we write down the product of univariate PDFs. And of course, these PDFs are given theta plug in different values of x. So it's a function of x with the value of theta held constant. Okay. Here's our likelihood function. Again, it's a function of theta given the data held constant. So if we look back to this little picture of our likelihood function, a logical thing to do would be follow this uphill and try to find the most likely value of theta. So we're wondering, what is this value here of theta that maximizes the likelihood function? So that value, that maximizer is called the maximum likelihood estimator. And we write mLE for short. All right. So our maximum likelihood estimator again is the value of theta that maximizes the likelihood function. All right. So if we have some experiences calculus and if this likelihood function is pretty nice, then we can just go ahead and use calculus to take derivatives and find the maximizer. For the stuff that we're doing in this class, it will be usually pretty much that simple as just using calculus. There are cases where it is harder to find the maximizer, but we'll either save that for later or not talk about it at all. Okay. So we're looking for this maximum likelihood estimator and that's again the maximizer of our likelihood function. And remember that log is a monotone function. And a lot of times actually we'll be talking about the log likelihood. And that's just the log of the likelihood function. And in this class, whenever we're talking about log, we always mean natural log. So we're using this log likelihood because it's a monotone function. So if we're maximizing the log likelihood, that's the same thing as maximizing the likelihood. And it's just a lot easier to use log because remember if we're taking the log of a product, that's just the sum of the logs. So it's a lot easier to take the derivative of a sum than is to take the derivative of a product like this. So the most of the time we want to write down the likelihood, then take the log and then we can take our first derivative, set that equal to zero, find that maximizer, and then take the second derivative to make sure that we actually have a maximizer and not a minimizer.