tomer7934/USA-RealEstate-Regression-RF

Assignment #2: Advanced Modeling & Feature Engineering (USA Real Estate)

Student Name: Tomer Naimi Course: Intro to Data Science

Video Presentation (Part 4)

https://youtu.be/B1WI04D2U5g

1. Data Integrity and Methodological Assessment (20%)

Critical Data Handling Decisions

Action	Analytical Justification	Implication
Extreme Outliers (Capping)	Applied Capping at 99.5th Percentile to all numerical features.	Methodological Necessity: Stabilized the training data against massive outliers ($1B Price, 473 Beds), which would otherwise collapse any predictive model.
Skewness Correction	Performed Log Transformation on the target variable (price).	Mitigated the strong positive skew in the price distribution, essential for meeting Linear Regression's underlying assumptions.
Feature Augmentation (FE)	Introduced Clustering Feature (KMeans ID) and One-Hot Encoding (OHE).	Crucial Step: Overcame the zero-correlation problem by injecting complex, non-linear location information into the model.
Data Integrity	Imputed ~16% missing values with Mode/Median.	Preserved statistical power by avoiding the massive loss of approximately 850 data points.

---

2. Model Training and Evaluation (20% & 20%)

Finding A: Regression Performance — The Power of Feature Engineering

Model Name	R-squared ($R^2$)	Score Improvement	Critical Insight
Baseline (Linear Regression)	0.3793	N/A	Failure Confirmed: Only 38% of price variance is linearly explained. Validated the need for advanced modeling.
Improved (Random Forest)	0.6019	$\mathbf{+58%}$ Increase in R2	The Winner: Captured non-linear interactions and proved the direct value of the Engineered Features (Clustering and OHE).

Finding B: Feature Interpretation & The Price Mechanism

Visualization: Bar Plot of Feature Importances

• Key Insight: The primary drivers of property value are not generic features, but Human Factors.
• Conclusion: The most significant predictors were bath (39% importance) and the zip_code (24% importance). This reinforces that price is determined by Convenience and Micro-Location rather than simple square footage.

Finding C: Classification Task Results — The Non-Learner

• Models: Logistic Regression / Decision Tree Classifier.
• Accuracy: $\approx 0.502$.
• Critical Evaluation: Despite perfectly balanced classes, the models failed to achieve any predictive power (equivalent to random guessing). The low F1 Score ($0.11$) and Recall ($\mathbf{0.06}$) on Class 1 (Expensive) demonstrate a catastrophic Failure to Differentiate high-value homes.
• Submission: The low performance justifies the submission of the Logistic Regression Model (as the baseline winner for this task) as a record of the model's limitations.

---

3. Final Submission Status

• Model Winner (Regression): random_forest_improved_model.pkl (R2: 0.6019)
• Model Winner (Classification): logistic_regression_model.pkl (F1: 0.11)
• Repository Content: Notebook, README, Data File, 4 Model files, 7 PNG visuals.