## 4 Experiment
### Model Training and Evaluation  
Given the comprehensive description of the training phase, several essential steps were taken throughout the learning process to guarantee accurate and reliable rental pricing projections for the ML models. Below is a detailed explanation of the procedure:  
**Data preparation**  
* Data cleaning: To improve the quality of the data, unnecessary observations, outliers, and inconsistencies were eliminated. K-nearest neighbours (KNNs) imputations was used to address missing numerical values, while missing categorical values were filled in using the fill-in approach. * Feature engineering: Building predictive models involves a critical stage called feature engineering, especially in the real estate industry, where the quality of features can significantly impact model performance. Two feature engineering approaches were used in this case. The One-hot encoding technique transforms categorical variables like \"newlyConst\", \"balcony\", \"Kitchen\", \"cellular\", \"lift\", and \"garden\" into numerical values for use in a ML model. Normalisation was applied to adjust features such as rental price, space, and range, ensuring these factors are on a comparable scale. * Feature selection: Recent research has highlighted the application of hybrid models and meta-heuristic algorithms, such as PSO and CatBoost. These two methods were applied to choose the most pertinent characteristics. While CatBoost reduced dimensionality by converting categorical data into TSs, PSO optimised feature selection using an objective function. **Model architecture**  
* A variety of models were investigated based on their demonstrated performance in comparable predicting tasks and their capacity to manage intricate connections within the data. The algorithms used encompassed LR, tree-based algorithms, neural networks, and stacked models. In terms of neural network structure, regularisation techniques such as dropout were applied to both the PNNs and FNNs to prevent overfitting and provide a fair comparison between these two algorithms. Dropout regularisation was used in the hidden layers with ReLU activation for both models. Moreover, both FNN and PNN models shared a similar architecture, consisting of two hidden layers with 256 neurons, respectively, and ReLU activation functions in both layers to ensure a fair comparison. The only difference was the framework used (TensorFlow for FNN and PyTorch for PNN); however, the overall model structure and hyperparameters were consistent across both frameworks. This provides clarity on the use of frameworks and ensures a direct comparison between models built with TensorFlow and those built with PyTorch. **Training process**  
* Training-validation split: An 80:20 ratio was used to divide the dataset into training and validation sets. This ratio ensures that there is enough data for both learning and evaluation the predicted accuracy of the models on new instances, with the goal of reducing overfitting and enhancing generalisation. The training set contained 3495 records, while the validation set included 874 items after the split. * Hyperparameter tuning: Key hyperparameters were methodically optimised across multiple rounds to improve model performance. These hyperparameters included the learning rate, batch size, number of layers, and number of neuron per layer. The study assessed learning rates between 0.00001 and 0.1, examined batch sizes ranging from 8 to 152, modified the number of layers from one to four, and varied the number of neurons per layer from 8 to 520. The optimal configuration was found to be a learning rate of 0.001, a batch size of 64, two layers, and 256 neurons per layer. * Epochs and iterations: A range of epochs, from 10 to 4000, was used to train the models. The models were continuously updated based on the error resulting from the difference between the actual and predicted values for each epoch. The best-performing epoch was found at 410 epochs.