asafmak/fifa_overall_predictor

FIFA Player Analysis – Regression, Feature Engineering & Classification

Presentation Video

Overview

This project analyzes FIFA player performance data and builds machine-learning models to predict and classify player ability.

The workflow includes:

• Data Cleaning & EDA
• Feature Engineering
• Regression Modeling
• Clustering
• Classification Modeling
• Exporting Winning Models

---

Dataset Description

Source: Kaggle – FIFA Players Dataset
Size: ~18,000 players
Features: Player attributes including age, height, weight, overall rating, potential, technical skills (passing, shooting, dribbling), physical abilities, positions, preferred foot, and market value/wage. Research Question

Which player attributes best predict a player’s overall rating, and how accurately can a model estimate this rating?

---

1. Data Cleaning & Exploration

Main steps:

• Removed columns with excessive missing values.
• Filled missing values (median).
• Looked for duplicate records.
• Verified no invalid values (e.g., negative height, negative weight).
• Identified numeric vs. categorical features.
• Visualized distributions, outliers, and correlations.

EDA: Understanding the Dataset Through Visuals

---

---

Purpose: Understand the data structure and detect patterns before modeling.

---

Research Questions

1. Is there a difference in the overall rating by playing position?*?

Insight: There are some differences in overall rating across positions, but the gaps are relatively small, and most positions fall within a similar overall range.

---

2. Do younger players have higher potential?*

Insight: Younger players tend to have slightly higher potential, though the relationship is weak and there is wide variation across ages.

---

3. Is salary related to the overall rating?

Insight: Defenders and goalkeepers tend to be taller, while attacking players and midfielders show more variance.

---

4. Are there differences in height between the different positions?

Insight: Defenders and goalkeepers tend to be taller, while attacking players and midfielders show more variance.

---

5. Does a player's preferred foot have an impact on their overall rating?

Insight: Right-footed and left-footed players have nearly identical distributions — preferred foot does not significantly impact player rating.

---

2. Baseline Regression Model (Linear Regression)

Goal: Predict a player’s overall rating using only the cleaned numerical features.

Steps Performed:

• Selected all relevant numerical features
  (after removing IDs, text fields, financial columns, and other irrelevant fields).
• Split the dataset into 80% train and 20% test.
• Built a Pipeline consisting of:
  • StandardScaler – normalizes all numeric inputs
  • Linear Regression – simple and interpretable baseline model
• Trained the model and evaluated performance using MAE, RMSE, and R².

Visualizations

1. Actual vs Predicted Overall Rating

This plot shows how close model predictions are to the true ratings.

---

2. Residuals Distribution

This shows how the prediction errors are distributed and whether the model is biased.

---

3. Feature Importance (Top Coefficients)

This highlights which numerical features most influence the model’s prediction.

---

Baseline Performance:

• MAE: ~1.60
• RMSE: ~2.05
• R²: ~0.916

Summary:

The baseline linear regression model performs strongly, explaining over 91% of the variance in player ratings.
This provides a reliable starting point before adding engineered features or more advanced models.

---

3. Feature Engineering

Added new features:

• Cluster ID from K-Means (unsupervised learning).
• Attack Index — averages offensive skills (finishing, shot power, dribbling…)
• Defense Index — averages defensive skills (tackles, marking…)
• Physical Index — averages physical abilities (strength, stamina, balance…)

These features capture broader skill profiles that individual columns cannot.

---

Choosing Number of Clusters

We compared multiple values of k using Silhouette Score and the Elbow Method.

• Silhouette indicated k = 2, but this split the players into overly large, uninformative groups.
• The Elbow Method showed improvement slowing around k = 4.

Final choice: k = 4, as it provides the best balance between structure and interpretability.

Description: The PCA plot shows that players form four distinct clusters based on their abilities.
Each cluster represents a different skill profile, which is why Cluster ID is a meaningful new feature for the model.

---

4. Improved Regression Model

I attempted to improve the Linear Regression model using the new engineered feature set, but its performance remained unchanged.
Linear Regression struggles to capture complex non-linear relationships in the data, so the additional features did not significantly improve its accuracy.

After this evaluation, I trained two additional models — Random Forest and Gradient Boosting — using the same improved features.
These models outperformed Linear Regression, with Random Forest achieving the best overall results.

Results:

Model	MAE	RMSE	R²
Improved Linear Regression	1.60	2.05	0.916
Random Forest	0.73	1.09	0.976
Gradient Boosting	0.86	1.19	0.972

🏆 Winning Regression Model: Random Forest

Random Forest won because it captured non-linear patterns better and achieved the lowest errors with the highest R².

---

5. Regression → Classification Conversion

Converted the continuous target (overall rating) into three balanced classes using quantile binning:

• Class 0 — bottom 33%
• Class 1 — middle 33%
• Class 2 — top 33%

---

6. Training & Evaluation Classification Models

To convert the regression task into classification, we trained three different classifiers (using the engineered features and the 3-class target):

1. Logistic Regression
2. Random Forest Classifier
3. Gradient Boosting Classifier

I evaluated the performance of all three classification models using the same test set to ensure a fair and consistent comparison.
We assessed each model using Accuracy, Precision, Recall, and F1-Score, and also generated confusion matrices to better understand the types of errors each model makes.

📊 Classification Model Performance

Model	Accuracy	Precision (macro)	Recall (macro)	F1 (macro)
Logistic Regression	0.845	0.847	0.844	0.845
Random Forest	0.933	0.932	0.931	0.931
Gradient Boosting	0.944	0.943	0.942	0.943

🏆 Winning Classification Model: Gradient Boosting

Gradient Boosting achieved the best performance across accuracy, precision, recall, and F1-score.

---

Project Summary

This project demonstrates that combining clean data, feature engineering, and clustering significantly improves player rating prediction.
Tree-based models outperformed Linear Regression, with Gradient Boosting achieving the strongest classification performance.
Overall, the optimized feature set and model comparisons produced robust and reliable predictive results.

---