Analyzing Melbourne House Prices with R | by Soner Yıldırım | Towards Data Science

There are several libraries and packages that provide data scientists, analysts, or any one interested in data with functions to perform efficient data analysis. Most of them are well-documented so you can easily find out what a function does.
However, the best way to learn such libraries is through practice. It is not enough to know what a function does. We should be able to recall and use them at the right time and place. Thus, I highly recommend to practice to learn a package or library.
In this article, we will use R packages to explore and gain insight into Melbourne housing dataset available on Kaggle.
For data analysis and manipulation, we will be using the data.table package of R. Let’s import it and read the csv file.
> library(data.table)> house_prices <- fread("Downloads/melb_data.csv")> head(house_prices)
The dataset contains several attributes of the houses in Melbourne along with their prices. Since the focus of this dataset is the price, it is better to get an overview of the price column first.
> house_prices[, summary(Price)]   Min.  1st Qu.  Median     Mean  3rd Qu.     Max.   85000  650000   903000  1075684  1330000  9000000
We use the summary function on the price column to get an overview in terms of basic statistics. The average house price is approximately 1.07 million.
We know the average house price in general. We might need to compare the house prices in different regions. This is a group by task and can easily be done by adding the name of column to be used for grouping.
> house_prices[, mean(Price), by = Regionname]
The aggregated column is represented as “V1” which is not very informative. We can assign a name to the aggregated column with a slight change in the syntax. Let’s also calculate the number of houses in each region along with the average house price.
> house_prices[, .(avg_price = mean(Price), number_of_houses = .N), by = Regionname]
There are 3 types of houses in the dataset which are h for house, u for unit, and t for town house. The data.table package also allows for filtering along with aggregating and grouping.
For instance, we can calculate the average land size of each house type in the northern metropolitan region.
> house_prices[Regionname == 'Northern Metropolitan', .(avg_size = mean(Landsize)), by = Type]   Type  avg_size1:    h  619.24912:    u  495.02653:    t  317.3257
I think it is better if we elaborate on the syntax structure used by data.table library.
datatable[ filter/order , select/aggregate/update , by ]
The expressions before the first comma are used for ordering or filtering. In the previous example, we filtered observations (i.e. rows) based on the region name column.
The second part is for aggregating, selecting, or updating. As you see in the examples, we have calculated the average prices in this part. The final part is where we group the rows according to the categories in the given column.
The structure of the syntax provides an efficient way to perform data manipulation, selection, and transformation. For instance, we can create a subset that only contains the address, number of rooms, price of the houses that cost less than 1 million.
> subset = house_prices[Price < 1000000, .(Address, Rooms, Price)]> head(subset)
We can also perform data transformation or aggregation while selecting the columns. For instance, we can represent the price in millions in the subset.
> subset = house_prices[     Price < 1000000,      .(Address, Rooms, Price_million = (Price/1000000))]> head(subset)
If we use the “keyby” parameter instead of the “by” parameter, the result set is sorted based on the categories increasing order. Consider the following example.
> house_prices[      ,.(avg_distance = mean(Distance), avg_price = mean(Price))       ,keyby =.(Type, Regionname)]
As we can see, the categories in the type and region name columns are sorted in ascending order.
The data.table package allows for chained operations which further simplify the syntax for more complex operations. Recall the previous example where we calculated the average distance and price of the houses for each type and region name combinations.
We may want to sort the groups according to the aggregated values such as average price. Here is how we can accomplish this task.
> house_prices[      ,.(avg_distance = mean(Distance), avg_price = mean(Price))       ,by =.(Type, Regionname)][order(avg_price)]
The order function sorts the values in ascending order by default. We can change it to descending order by adding a minus sign in front of the column name (e.g. order(-avg_price)).
We have done several examples to demonstrate the simple and efficient methods of the data.table package for data analysis and manipulation.
It makes the operations more intuitive to be able to combine filtering, selecting, transforming, and aggregation into a single square brackets.
There is much more to cover in the data.table package but what we have covered in this article provides a decent set of operations for efficient data analysis.
Thank you for reading. Please let me know if you have any feedback.