# The Impacts of Open Data and eXplainable AI on Real Estate Price Predictions in Smart Cities
## 2 Materials and Methods
### Data
#### 2.1.3 Descriptive Statistics  
Before deploying our machine learning models, we first preprocessed the data. During this stage, we addressed missing values within the dataset, particularly for the EPC feature, which had a 12.05% missing rate. We used a simple imputation method, replacing missing values with the mode to preserve the feature's ordinal nature. Conversely, features with a high percentage of missing values, such as Garage, Pool, Terrace, and Patio, were excluded from our analysis. This decision was due to their excessively high missing rates, with the Garage at 30.14% and the Pool, Terrace, and Patio all sharing a missing rate of 34.07%. Excluding these variables prevents potential distortion in the predictive model outcomes [68]. Our approach included detecting and handling outliers in the target variable price, recognizing that outliers can significantly impact model performance and generalizability, particularly in regression models. Outliers can distort a model's generalization ability by leading it to overfit these extreme values. Therefore, we applied the Modified Z-score method to identify individual outliers in a single variable [69]. This method, relying on the median and median absolute deviation (MAD), is particularly effective for non-normally distributed data, such as the right-skewed distribution of the price feature. The Modified Z-score method, with a threshold of 3.5, was used to identify and remove 1204 outliers from the price feature that could skew our analysis. In addition to this univariate treatment, we performed a multivariate outlier inspection by considering PGA (private gross area), the feature most highly correlated with price. We specifically targeted unusual combinations of values across these two features that are unlikely to be encountered in real-world deployment scenarios. In this process, we identified and removed observations with abnormally low prices per square meter, specifically those under EUR 1000/m\({}^{2}\). This analysis resulted in the exclusion of 107 observations from our dataset. By treating outliers, we refined our dataset to 22,470 records, retaining only those real estate transactions that most typically reflect the broader market trends and represent the majority of the population of our dataset. Our study employed ArcGIS Pro (version 3.2.2) within a Geographic Information System (GIS) framework to analyze the impact of urban amenities and infrastructure accessibility on patterns of real estate transactions. The preprocessing of open data features was approached using two methods. First, we quantified the number of points of interest (POIs)--such as Trees, Bus stops, and Schools--within each property listing's 75, 250, and 500 m radii, respectively. This process involved creating spatial buffers around the properties and afterward counting the POIs within these areas to assess the density of amenities. Second, we calculated the distance from each property to the nearest POI categories, including Commerce, Culture, Health, Parks, Safety, Security, Sports, Subway, Train, and University facilities, using proximity calculations to establish potential accessibility. The outlined methodology is designed to elucidate the spatial dynamics between property transactions and the attractiveness of various urban zones, providing insights critical for real estate valuation and urban development planning. To manage the issue of high dimensionality and the problems with one-hot encodings, like increased sparsity and multicollinearity, we converted 7-digit postal codes into a numerical format and binary encoded the property condition feature (PropCond). Feature selection was conducted using Recursive Feature Elimination (RFE) in conjunction with feature importance scores from XGBoost models and correlation matrices. We partitioned our dataset into two segments, using 80% for training purposes and the remaining 20% for testing, in line with the standard train-test split methodology.