A Guide to Machine Learning in R for Beginners: Decision Trees
By: Parul Pandey
Tags: Decision Tree, Machine Learning, Data Science, R Language

Decision Trees in R

We will be working on the famous Boston housing dataset.This data comes from a paper, “Hedonic Housing Prices and the Demand for Clean Air” exploring the relationship between prices and clean air in the late 1970s.We will explore the boston.csv data set with the aid of trees. Here we are interested in building a model initially of how prices vary by location across a region.

Dataset

We will explore the boston.csv data set with the aid of trees. Download this file from here to follow along. Each entry of the dataset corresponds to a census tract As a result, there are multiple census tracts :

LON and LAT are the longitude and latitude of the center of the census tract.

MEDV is the median value of owner-occupied homes, measured

in thousands of dollars.

CRIM is the per capita crime rate.

ZN is related to how much of the land is zoned for large residential properties.

INDUS is the proportion of the area used for industry.

CHAS is 1 if a census tract is next to the Charles River else 0

NOX is the concentration of nitrous oxides in the air, a measure of air pollution.

RM is the average number of rooms per dwelling.

AGE is the proportion of owner-occupied units built before 1940.

DIS is a measure of how far the tract is from centres of employment in Boston.

RAD is a measure of closeness to important highways.

TAX is the property tax per $10,000 of value.

PTRATIO is the pupil to teacher ratio by town.

here MEDV is the output /target variable i.e price of the house to be predicted. Since the output variable is continuous it is an example of a regression tree.

Working

1. Analyzing the Data

Loading data into R console:

> boston = read.csv('boston.csv') > str(boston) 'data.frame': 506 obs. of 16 variables:

$ TOWN : Factor w/ 92 levels "Arlington","Ashland",..: 54 77 77 46 46 46 69 69 69 69 ...

$ TRACT : int 2011 2021 2022 2031 2032 2033 2041 2042 2043 2044 ...

$ LON : num -71 -71 -70.9 -70.9 -70.9 ...

$ LAT : num 42.3 42.3 42.3 42.3 42.3 ...

$ MEDV : num 24 21.6 34.7 33.4 36.2 28.7 22.9 22.1 16.5 18.9 ...

$ CRIM : num 0.00632 0.02731 0.02729 0.03237 0.06905 ...

$ ZN : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...

$ INDUS : num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87 ...

$ CHAS : int 0 0 0 0 0 0 0 0 0 0 ...

$ NOX : num 0.538 0.469 0.469 0.458 0.458 0.458 0.524 0.524 0.524 0.524 ...

$ RM : num 6.58 6.42 7.18 7 7.15 ...

$ AGE : num 65.2 78.9 61.1 45.8 54.2 58.7 66.6 96.1 100 85.9 ...

$ DIS : num 4.09 4.97 4.97 6.06 6.06 ...

$ RAD : int 1 2 2 3 3 3 5 5 5 5 ...

$ TAX : int 296 242 242 222 222 222 311 311 311 311 ...

$ PTRATIO: num 15.3 17.8 17.8 18.7 18.7 18.7 15.2 15.2 15.2 15.2 ...

There are 506 observations corresponding to 506 census tracts in the Greater Boston area. We are interested in building a model of how prices vary by location across a region. So, let’s first see how the points are laid out.

Using the plot commands, we can plot the latitude and longitude

of each of our census tracts.

Let’s first see how the points are laid out using the plot commands.

# Plot observations

> plot(boston$LON, boston$LAT)

The dense central core of points

corresponds to Boston city, and other urban cities.

Since, we also have the Charles river attribute(CHAS), we want to also show all the points that lie along the Charles River in a blue colour.



We can do this by the points command.

# Tracts alongside the Charles River > points(boston$LON[boston$CHAS==1], boston$LAT[boston$CHAS==1], col="blue", pch=19)

Now we have plotted the tracks in Boston along Charles river.

What other things can we do?

Well, this data set was originally constructed to investigate questions about how air pollution affects prices.

The air pollution variable in the data is NOX. Let’s have a look at a distribution of NOX.

# Plot pollution/NOX > summary(boston$NOX) Min. 1st Qu. Median Mean 3rd Qu. Max.

0.3850 0.4490 0.5380 0.5547 0.6240 0.8710

The minimum value is 0.385 and the maximum value is 0.87 .The median

and the mean are about 0.53, 0.55. So, let’s just use the value of 0.55,

as it is the centre most value.

Let’s look at the tracts that have above-average pollution.

> points(boston$LON[boston$NOX>=0.55], boston$LAT[boston$NOX>=0.55], col="green", pch=20)

All the points that have got above-average pollution are coloured green.

Now it kind of makes sense, since, the area most densely polluted is the one which is also most densely populated.

Now let us look at how the prices vary over the area as well. We can do this with the help of MEDV variable using the same methodology as done when plotting the pollution.

# Plot prices

> plot(boston$LON, boston$LAT) > summary(boston$MEDV)

Min. 1st Qu. Median Mean 3rd Qu. Max.

5.00 17.02 21.20 22.53 25.00 50.00 > points(boston$LON[boston$MEDV>=21.2], boston$LAT[boston$MEDV>=21.2], col="red", pch=20)

above median values graph

So what we see now are all the census tracts with above-average housing prices in red.

However, the census tracts of above-average and below-average are mixed in between each other.

But there are some patterns.

For example, look at that dense black bit in the middle. That corresponds to most of the city of Boston, especially the southern parts of the city.

So there’s definitely some structure to it, but it’s certainly not simple in relation to latitude and longitude at least.

2. Applying Linear Regression to the problem

Since, this is a regression problem as the target value to be predicted is continuous(house price), it is but natural that we look up to Linear Regression algorithm to solve the problem. We saw in the last graph that the house prices

were distributed over the area in an interesting way, certainly not the kind of linear way and we feel Linear Regression is not going to work very well here. Lets backup our intuition with facts.

Here we are plotting the relationship between latitude and house

prices and the longitude and the house prices, which looks pretty nonlinear.