An Analysis of Property Prices in Ireland
By: Chirag Chadha
Tags: Python, Housing, Data Analysis, Data Science, Data Visualization

Processing the data

All records can be downloaded from the Property Price Register (PPR) site in CSV format. In this article, we’ll take a look at the last two years of data. It should be noted that the PPR page contains a disclaimer stating:

The PSRA does not in any way edit the data. It simply publishes, in a fully transparent manner, the data from the declarations which are filed with the Revenue Commissioners. If the data filed contained typographical errors then those errors will appear on the Register.

Let’s take a look at some steps that were taken to clean up the raw information using Pandas:

Remove properties with prices that were not full market price and remove rows that contained a small amount of Irish language descriptions of properties.

df = df[df['Not Full Market Price'] == 'No'] df = df.drop('Not Full Market Price', axis=1) df = df[(df['Description of Property'] == 'Second-Hand Dwelling house /Apartment') | (df['Description of Property'] == 'New Dwelling house /Apartment')]

2. Look at a subset of the data from 2017 onwards.

df['Date of Sale (dd/mm/yyyy)'] = pd.to_datetime(df['Date of Sale (dd/mm/yyyy)'], dayfirst=True, format='%d/%m/%Y') df = df.rename(columns={'Date of Sale (dd/mm/yyyy)': 'Date of Sale', 'Price ()': 'Price'}) df = df[(df['Date of Sale'] > '2017-01-01')].reset_index(drop=True)

3. Parse the price column from strings into floats.

df['Price'] = df['Price'].apply(lambda x: x.lstrip('\x80')) df['Price'] = df['Price'].apply(lambda x: float(x.split()[0].replace(',', ''))).astype(float)

4. Attempted geocoding using Google’s API to obtain latitude and longitude of properties.

In the interest of clarity, I should note that the coordinates I obtained for the last two years of PPR data turned out to be inaccurate enough that I’m not including the results in this piece. It is likely that the fault lies in the recorded addresses contained in the data (which are self-reported and unedited by the PRSA) and not in Google’s actual API. The accuracy could probably be improved if the county of each property was attached to the end of the address (some addresses contain the county but some do not). Regardless of the inaccuracies, I felt it was important to include the code written to request the coordinates.

The GET request url creator function:

def url_creator(address, url='https://maps.googleapis.com/maps/api/geocode/json?address=', api_key=''):

logger.debug('Creating API request URL')

add_list = address.split()

add_list = [line + '+' for line in add_list[:-1]]

add_list.append(address.split()[-1])

add_url = "".join(add_list)

return url + add_url + '&key=' + api_key

The latitude and longitude request function:

def lat_lng(address, api_key=''):

logger.debug('Requesting geospatial info from Google API')

url = url_creator(address, api_key=api_key)

try:

response = requests.get(url)

response.raise_for_status()

except requests.exceptions.HTTPError as e:

logger.error(e)



results = response.json()['results']

if len(results) != 0:

logger.debug('Found geospatial coordinates')

return results[0]['geometry']['location']['lat'], results[0]['geometry']['location']['lng']

else:

logger.debug('Could not find geospatial coordinates')

return np.nan, np.nan

The reason I wrote my own code to do this stuff (and more) with the Geocoding API instead of using the popular geocoder library is because of the outdated quota behaviours for the API imposed in the library. Google has recently made it possible to make an unlimited number of requests to its Geocoding API but geocoder’s Google geocoding method seems to have a limit set at 2500 API calls. If you’re interested, I’ve included all of the code I used for processing the data and geocoding in the GitHub repo linked at the end of this article.