Predicting Apartment Rental Prices in Germany
By: Vineeth Antony
Tags: Exploratory Data Analysis, Germany, Machine Learning, Housing Prices, Data Science

Through this article, I have attempted to predict rental prices using the apartment rental dataset containing rental prices in Germany. The dataset consists of data scraped from one of Germany’s biggest real-estate platform. The main objective here is to study and understand the data and use the knowledge to construct a basic predictive model to predict the base rental price (popularly known in Germany as ‘Kaltmiete’).

The original dataset consists of 268850 apartments (rows) * 49 features (columns), but a combination of missing and imbalanced values means that only 14 features were useful to me in the current context. The columns contain the following information:

regio1 : The federal state in which the apartment is located. heatingType: The type of heating system used in the apartment. balcony: Column indicating if the apartment has a balcony. yearConstructed: The year in which the apartment was constructed. hasKitchen: Column indicating if the apartment has a kitchen. cellar: Column indicating if the apartment has a cellar. baseRent: The rent excluding electricity and heating. livingSpace: The living space in square meter. condition: The condition of the flat. lift: Column indicating if the apartment has a lift. typeOfFlat: The type of apartment. noRooms: Total number of rooms in the apartment. garden: Column indicating if the apartment has a garden. regio2: The city where the apartment is located.

Dataframe at a glance(1/2)

Dataframe at a glance(2/2)

Exploratory Data Analysis:

Since this dataset was created by web scraping, the dataset was far from clean. Therefore one of the challenges was getting rid of extreme and null values. For example, there were several rows with a living space of more than 60 sq. m. in the range of €10-€30, which are clear outliers. There were also several apartments with the total number of rooms more than 100. A post-cleanup study yielded the following:

1.) The city in Germany with the highest average rent prices is Munich, which is not surprising as the city is quite well known for its notoriously high living costs. The districts in and around Munich (denoted by München_Kreis) also figure in the Top 10. Interesting fact, Starnberg, which follows Frankfurt at number three, is popularly known as the wealthiest town in Germany. I was quite surprised not to find Düsseldorf and Bonn on the list.

Average Rents by Cities

2.) Hamburg is the federal state with the highest average rental prices, followed by Berlin and Bayern(Bavaria). The North-Eastern states of Sachsen-Anhalt and Thüringen have the lowest prices.

Average Rents by State

3.) As should be, the size of the living space and the base rental price seems to be more or less positively correlated. The small(in size) apartments with high prices that seem like outliers are apartments located in the city center (central Berlin, Frankfurt) which explains the cost.

Total Rent vs Living Space

4.) The most common base rental price in Germany hovers around the €300-€400 range.

Rent Distribution in Germany

5.) Central Heating seems to be the most common type of heating system, by a fair margin, employed in German homes.

6.) Most of the apartments on offer have 3 rooms and come under the type ‘apartment’.

Count of Rooms per apartment

Type of Rentals

Model Development:

Before a prototype model can be created, it is important to transform the dataset. The following pre-processing steps were performed on the dataset before being fed into the model.

The seven unique values in the ‘condition’ column (‘well_kept’, ‘refurbished’, ‘first_time_use’, ‘fully_renovated’, ‘mint_condition’,

‘first_time_use_after_refurbishment’, ‘modernized’, ‘negotiable’,

‘need_of_renovation’, ‘ripe_for_demolition ') were divided into 3 classes ‘new’, ‘old’ and ‘middle’ apartments.This was done to get a clearer demarcation. The data in the columns ‘balcony’, ‘kitchen’, ‘cellar’, ‘lift’ and ‘garden’ were encoded to 1 or 0 depending on whether the column value was True or False. The continuous-valued variables ‘baseRent’, ‘livingSpace’ and ‘noRooms’ were normalized using sklearn’s MinMaxScaler(). While this is not a necessary step when dealing with tree-based algorithms, it eases interpretability and makes performance comparison with other models easier. The categorical variables were encoded using One-Hot-Encoding, made easy by pandas’ get_dummies. The column ‘baseRent’ was selected as the target variable and the rest as predictor variables. After applying the above steps, the dataset was split into a training set(70% of the data) and testing set(30% of the data) using sklearn’s train_test_split.

I chose Random Forest to develop the prototype as it is a powerful model that can work well with both continuous and categorical variables.

The model was fit using RandomForestRegressor() on the training set. Using the test set to check the performance of the model, the following results were obtained.

Model Results

The model accounts for around 83.4% of the data in the model or the predictive power of the model is 0.83 (best is 1.0), which is quite good considering that the model was run using the default parameters and not tuned. The MSE and MAE are also on the lower side.

Conclusion:

Can this model be improved? Yes, of course!. The model is very far from being the best model. My first approach would be to perform Hyperparameter tuning and cross-validation using grid-search techniques to find the ideal parameters. Secondly, the original dataset contained a lot of columns with missing data which I chose to exclude from the model, which meant quite some information was lost. Smart Imputation techniques using domain knowledge could help tackle this problem. The original dataset also contained columns describing apartment facilities and descriptions. By employing NLP techniques, more information can be gained from these columns. Heavier feature engineering would also be helpful. Needless to say, I will be using these steps as cues to better the performance of the model.

I would love to hear your feedback on this. If you have any questions or suggestions you can reach out to me. Thank you for reading.