Exploratory Data Analysis(EDA) on Residential Properties in Hyderabad
By: Bhargava Sai Reddy P
Tags: Eda, Python, Data Analysis, Data Science, Data Visualization

This article is a follow up of the previous one where I built a web scraper that extracted required data for a real estate website for analysis of residential properties in Hyderabad. In this, our primary focus would be on visualizing the data we have extracted and get meaningful insights of data so that we can come to the necessary conclusions.

Our ultimate goal is to find the residential properties with the requirements of my father which are shown stated below which states that there are four requirements i.e, price to be between 25 to 40 lakhs, the area should be greater than 1000 sqft, the property should be a 2 or 3 bhk apartment which is located in the areas kukatpally, gachibowli and miyapur.

We start with basic exploration and visualization of the data we have. For performing EDA we have univariate analysis, bivariate and multivariate analysis.

Firstly, we import the necessary libraries and see the basic details about the data like the number of rows, columns and the data types of each column. (rows represent the data points and column represents the features/attributes of data).

Let’s take a look at the top and bottom rows of the dataset. To get the data types of each column we can use df.dtypes which returns the datatypes of each column. (df is the data frame we loaded)

Now let’s visualize every column we have,

This bar plot above shows residential plots are the type of properties that are mostly found in Hyderabad which are about 14642 followed by 3 bhk apartments to be 4071 and similarly the count of other residential properties.

The above plot depicts that there 4 categories in building_status which are New properties to be almost 12000, Under construction properties to be almost 5000, Ready to move properties to be 5800 and Resale type of properties to be 1900. To get the exact count of each category we can use value_counts().

Now let’s move to our main task which is to find properties with mentioned requirements as above.

Specified Price range 25 to 40 L

when we consider the price range 25 to 40 L number of rows are reduced from 25715 to 4789 that means there are only 4789 properties in this price range.

Required locations: Kukatpally, Gachibowli, Miyapur

When we consider the mentioned locations we are left with only 85 properties.

Requires only Apartment’s

To find the properties that are apartments we need to set the index as the title so that we can fetch the required data and then reset the index. We have only 69 properties that are apartments.

Do you want to know what is the average price of the property? If so we have a savior describe() function which returns the mean, median, std, min value, max value, 25th 50th & 75th percentiles of data etc.

Here count represents the number of rows and mean represents the average value of each column,25% says that 25% of data has a value less than that mentioned value there for example for column price 25% says that 25% of properties have their price less than 30.12 L.The minimum price of property is 25 L and maximum price is 40 L. And 75% of properties have their prices less than 37 L.

What say? Let’s start visualizing??

Univariate Analysis

This a pie plot for the price column it shows that 20% of properties have their price as 40 L and 17.50% properties to have 25 L. Each pie in the plot represents different percentages of prices.

Let’s see the distribution of prices.

The left plot is a histogram in which x-axis represents the price and the y-axis represents the count/frequency of that price. The plot that is to the right is a distribution plot where the x-axis represents the price and the y-axis represents the probabilities of prices. For example, almost 23 properties are having a price range of 34 to 35 L and almost a probability of 0.12 for having a price range of 34 to 35 L.

In the same way, we can plot for other columns as well as shown below.

Plot for rate_persqft and area_insqft

Plot for building_status

Plot for locations

Oh! I guess we have forgotten to include two more requirements that are area and type of apartments. Well, no problem lets include them now and start further analysis and visualizations.

So after further adding our conditions we are only left with 46 properties.

Bivariate Analysis

Plot for Title vs Price(L)

The above figure is a box blot in which the two lines on both ends are called whiskers, the bottom line is the minimum value where the topmost line is the maximum value for each category i.e, 2 bhk and 3 bhk and the middle line inside the colored area specify the median which is also the 50th percentile. The plot describes that for 2 bhk the minimum price is 25 L and maximum price is 40 L, 25% of 2 bhk’s has their prices below 29 L whereas in 3 bhk the minimum starts at around 35 L and goes up to 40 L where 25% of 3 bhk’s has their price below 37 L.

A Joint plot between area_insqft and price(L)

From the above plot we can infer the relation between area and price by seeing the histograms on top of each axis of the plot. Surprisingly there is no increase in price with an increase in the area of the property(these assumptions are only for the data that we have got after applying the conditions that are specified)

Boxplot between location and price(L)

Multivariate Analysis

We can see from the above plot that miyapur has a wide range of prices from 25 L to 40 L in which most of them are 2 bhks. Whereas kukatpally has very few apartments whose price range starts from 35 L to 40 L. In gachibowli there are only two 2 bhk’s whose prices are 33 L and 40 L.

This is a violin plot which describes that miyapur has both types of apartments that are ready to move and under construction as well where most of the ready to move apartments have their price between 35 L and 36 L. And gachibowli has only under construction apartments.

This a scatter plot where the x-axis is area_insqft and the y-axis is rate_persqft. We can see that most of the apartments in miyapur has a rate per sqft between 2000 and 2500, whereas in kukatpally rate per sqft is mostly between 3000 and 3750.

Correlation: It defines the relationship between two features whether they are positively correlated or negatively correlated. Result of a correlation is called the correlation coefficient (or “r”).It ranges from -1.0 to +1.0. The closer r is to +1 or -1, the more closely the two variables are related. If r is close to 0, it means there is no relationship between the variables. If r is positive, it means that as one variable gets larger the other gets larger. If r is negative it means that as one gets larger, the other gets smaller (often called an “inverse” correlation).

This is called a correlation matrix. We can even visualize it using seaborn.

We can also use pair plots for correlation.

For the price range considered the highest correlation is between rate_persqft and price, they both have a positive correlation. Negative correlation exists between area_insqft and rate_persqft. There’s not much correlation between price and area_insqft.

Map Visualization

We can also use maps for visualization using a folium package.

importing necessary libraries

To get a map visualization we need latitude and longitude of the locations. Let us consider the three locations that were specified as one of the requirements(miyapur, kukatpally, gachibowli). And we will write a function that returns latitude and longitude of a location using the geopy module.

From above we can see that we were able to extract latitude and longitude of a location. Now let’s try plotting them.

Let’s try plotting for all the locations we have in the dataset. we can extract all the latitudes and longitudes of locations in the same way as we have done above. First, extract all the lat n long of locations and then create two new columns in our original dataset latitude and longitude and store extracted values there and then do as follows,

The above map shows us the intensity of properties in different locations of Hyderabad.

In conclusion, there is a total of 14642 residential plots, 8381 Apartments, 4071 3bhk apartments, 832 2bhk independent houses,440 3bhk villas, etc. And in consideration to conditions give we finally have 46 properties that suit the requirements mentioned. Almost 20% of properties have a price of 40 L in price range 25 to 40 L and miyapur has more number of residential properties.