Machine Learning: Model Representation And Hypothesis
By: Xuankhanh Nguyen
Tags: Machine Learning, Data Science, Deep Learning, Stats, Coursera

First, the goal of most machine learning algorithms is to construct a model or a hypothesis. In machine learning, a model can be a mathematical representation of a real-world process. To generate a machine learning model we will need to provide training data to a machine-learning algorithm to learn from.

Fundamental Segmentation of Machine Learning Models

All machine learning models are categorized as either supervised or unsupervised. If the model is supervised, it’s then sub-categorized as either a regression or classification model.

Supervised learning involves learning a function that maps an input to an output based on example input-output pairs — that is, given data with label (i.e. right answer)

Supervised learning problems can be further grouped into regression and classification problems.

Classification: A classification problem is when the output variable is a category, such as pass or fail, red or blue, etc. To illustrate, here we have training data with tumor size as its category which represents a feature and label respectively.

source: slide from lecture 1

In this example, we labeled 0 as benign and labeled 1 as malignant. Now we wish to make a surprised model. When new data comes in, our trained model predicts its label, that is, label 1 (malignant) or label 0 (benign). This is an example of a classification problem — given some possibly continuous input, we want to predict a discrete output.

Regression: A regression problem is when the output variable has a real value, such as weight, height, or dollars. To illustrate, here we have training data with housing size and price, which represents a feature and label respectively. When new data comes in, our goal is to predict its label, that is, the corresponding value of the housing price, in the range [0, 400]. This is an example of a regression problem — given some possibly discrete input, we want to predict a continuous output.

slide from lecture 1, week 1

Note:

Binary or Binomial Classification are used when we group data using two kinds of labels.

Multi-class or Multinomial Classification are used when we group data using more than two kinds of labels.

Unsupervised learning is used to draw inferences and find patterns from input data without references to labeled outcomes — that is, given data without any label (i.e without any right answer)

Unsupervised machine learning purports to uncover unknown patterns in data but most of the time these patterns are poor approximations of what supervised machine learning can achieve.

Supervise Learning vs. Unsupervised Learning

The biggest difference between supervised and unsupervised learning is that each data have a label in the case of supervised learning, whereas there is NO label for each data in the case of unsupervised learning, implying, they have not been classified.

source: slide from lecture 1, week 1

Note:

Supervised learning will always have an input-output pair.

Unsupervised learning is just data without a label nor meaning that we try to make some sense out of it.

We use unsupervised machine learning when we do not have data on desired outcomes, such as determining a target market for a new product that a business has never sold before. However, if we are trying to get a better understanding of our existing consumer base then supervised learning is the optimal technique.

Making a Model with Supervised Learning

So far, we learned that the primary goal of most of the machine learning algorithms is constructing a model to make predictions. We can restate this model as a hypothesis.

The hypothesis basically maps an input to an output. An input variable refers to a feature and an output variable refers to a target. The dataset we will be using to learn out model is called a training set.

Our goal is, given a training set, to learn a function h : X → Y so that h(x) is a “good” predictor for the corresponding value of y.

source: Slide from lecture 2, week 1

The figure above diagrammatically represents model h(x). So, let's say, we train a model based on a bunch of housing data that includes the size of the house and the sale price. By training a model, we have developed a model that will estimate the output (Y) — the price of houses based on our input(X) — the size of the houses. This is an example of linear regression.

Linear regression performs the task of predicting a target y based on given features x. So, this regression technique finds a linear relationship between x (input) and y(output).

The hypothesis is usually presented as

The theta values are the parameters

Linear regression gives us a straight line that best fits the data points. Let’s visualize the hypothesis with some examples.

This yields h(x) = 1.5 + 0x. 0x means no slope and y will always be the constant 1.5. This looks like:

h(x) = 1.5 + 0x

let’s see another example:

h(x) = 1 + 0.5x

The goal of creating a model is to choose parameters or theta values so that h(x) is close to y for the training data, x and y.

Wrapping up, we’ve learned about supervised learning, unsupervised learning, and linear regression. In the next article, we will talk about the Cost Function.

Keep on, keeping on!

Resources:

Unsurprisingly, there are many resources on Model Regression. However, not all of them are necessarily easy to understand! Some of them dive deep into the theory and mathematics of the problem, and it can be hard to find beginner-friendly content. Here are a few resources that are fairly easy to understand, and a good place to get started if you’re looking to learn more: