So far this semester we've been talking about unsupervised and supervised learning methods, and these really represent two ends of a spectrum. With unsupervised learning, we only have samples from some sort of input feature space. With supervised learning, these samples are also labeled in some way. So we might have a class label or some sort of a continuous value that we've coupled with each of those samples. For many problems that are out there, observations are very easy for us to collect, but actually labeling those samples can be very difficult. So for example, it might just be very expensive to provide those labels. We might have to hire a domain expert to look at each of our examples and make a determination as to what the correct label is. And this might be feasible to do if we have 100 samples or a couple hundred samples. But once we start to get into the tens of thousands, hundreds of thousands, or millions of samples, unless we're hiring a tremendous number of people, this is a very infeasible way to go. So one possibility is that we just ignore all of this other data that's sitting out there and just focus on the samples that are labeled, and then apply our standard supervised learning methods and move on from there. Alternatively, we can start to think in terms of semi-supervised learning. So in this case, we have a set of samples in our input feature space, and some of them are labeled and some are not. And for those that are labeled, if we look at their distribution in the input feature space, that set might not be terribly representative of the true distribution that's out in the real world. And these additional unlabeled features can actually give us a much better picture of what that true distribution is. So the idea is to try and make use of the entire set of samples to get that picture of what that distribution really is. And then from there, begin to apply our supervised learning methods, perhaps just with the set of samples that are labeled. There are both classification and supervised learning types of approaches that we can take for classification. What we end up doing essentially is making some inferences about what the labels might be for those samples that are unlabeled. So way to go about that is to first construct a model just with the labeled samples. And then we turn around and use that model to provide these pseudo labels to the remaining set of samples. And then from there, we can construct a more comprehensive model that makes use of both the labels and the pseudo labels. On the regression side of things, we've already talked about making use of some sort of a compression method as a pre processing step to our input features. So this is principal component analysis. This is iso map what have you. And once we do the compression with either any of these methods, then we apply supervised learning in order to construct our models. But the key here is that that first compression step, the learning process only makes use of the input samples and not the actual labels. So what we can do is actually make use of both labeled and unlabeled data to compute that compression. And then the idea is to learn the last step that regression step using just the labeled data. So over the next couple of videos, we're going to look both at the classification side as well as the regression side.