VIDEO:
https://drive.google.com/file/d/1KLCDMq2n9C69wWqUUUAh40RjqUMImmXb/view?usp=drive_link
Exploratory Data Analysis (EDA) Data Cleaning
The dataset contains 53,949 player-season records from 49 professional basketball leagues. Several columns included large amounts of missing data (such as high_school and all draft-related fields), which provided limited analytical value. These columns were removed. Text-based height and weight fields were also dropped, as numeric versions (height_cm and weight_kg) were already available.
Missing values in Team and nationality were filled with the category "Unknown". Rows with missing height_cm or weight_kg were removed to ensure consistent physical attribute data. Players with zero games played (GP = 0) or zero minutes played (MIN = 0) were excluded, as these records do not represent meaningful on-court performance.
An additional feature, AGE, was engineered using the player's birth year and the season year extracted from the Season column.
After cleaning, the dataset remained large, complete, and suitable for modeling.
Outlier Handling
Extreme performance values (e.g., very high points, assists, or rebounds) were retained because they represent legitimate variations in player performance. Only physically implausible or non-informative values were removed, such as unrealistic height or weight ranges and records with zero playing time. This ensured that the cleaned dataset reflects real basketball performance without artificial distortion.
Descriptive Statistics
Basic statistical summaries (mean, median, standard deviation, min/max) were computed for key performance metrics such as points (PTS), assists (AST), minutes (MIN), rebounds (REB), and physical attributes. A Pearson correlation analysis between PTS and AST was also performed to provide an initial indication of the relationship between scoring and playmaking.
Research Questions and Visual Insights Research Question 1
Does a higher number of assists (AST) correlate with higher or lower scoring (PTS)?
Insight: There is a clear positive relationship between assists and points. Players who accumulate more assists tend to score more points on average. The LOWESS trendline shows an upward trajectory throughout the AST range, indicating that increased playmaking activity is associated with higher scoring output.
Visualization:
Research Question 2
Does player height (cm) influence rebounding performance (REB)?
Insight: Height has a strong and consistent positive effect on rebounding. Taller players show significantly higher rebound totals, and the trendline rises sharply with height. This is consistent with expected physical advantages in basketball.
Visualization:
Research Question 3
Does player weight (kg) influence the number of rebounds (REB)?
Insight: Weight shows a mild positive relationship with rebounds, but the effect is considerably weaker than height. The trendline increases gradually, and the scatter is wide, suggesting that weight alone is not a strong predictor of rebounding. Other factors such as positioning, athleticism, and role appear to play a larger role.
Visualization:
Regression Goal
The goal of this regression task is to predict the number of points a basketball player scores in a given season (PTS), using the physical attributes and performance statistics available for that same season. This is a supervised regression problem in which PTS serves as a continuous target variable, and the model aims to learn the relationship between features such as assists, rebounds, minutes played, steals, turnovers, height, weight, and age, and the player's overall scoring output. Establishing this baseline model will help evaluate which features contribute most to scoring performance and provide a foundation for more advanced modeling in later stages.
Feature Selection
To construct the baseline regression model, I selected all meaningful numerical and categorical features that may reasonably contribute to predicting a player's scoring output (PTS). Irrelevant identifiers such as player names, textual descriptions, and draft information with extensive missing values were excluded. The selected features include season-level performance statistics (e.g., assists, rebounds, minutes played, steals, blocks, turnovers), physical attributes (height, weight, age), and categorical variables such as League and Team, which will later be encoded. This initial feature set provides a broad foundation for the baseline model, ensuring that the Linear Regression model has access to all potentially informative variables before applying any advanced feature engineering or dimensionality reduction techniques.
Train–Test Split
To evaluate the baseline regression model, the dataset was randomly partitioned into training and testing subsets using an 80/20 split. A fixed random seed (random_state=42) was applied to ensure reproducibility. The training set is used to fit the Linear Regression model, while the test set provides an unbiased evaluation of the model’s performance.
Model Training - Baseline Linear Regression
For the baseline model, I used a standard Linear Regression from scikit-learn with default parameters. goal at this stage is not to optimize performance, but to establish a simple, interpretable benchmark for predicting the number of points (PTS) a player scores in a given season.
The input features (X) include season-level performance statistics (such as assists, rebounds, minutes played, field goal attempts and makes, three-point attempts and makes, free-throw attempts and makes, steals, blocks, turnovers, and fouls), as well as physical attributes (height, weight, age) and categorical variables (League and Team).
Since the dataset contains both numerical and categorical features, a preprocessing step was applied using a ColumnTransformer: numerical features were passed through as-is, while categorical variables (League and Team) were one-hot encoded. This preprocessing was combined with the Linear Regression model in a single Pipeline, which was then trained on the training set (X_train, y_train).
Model Evaluation
To evaluate the performance of the baseline Linear Regression model, we measured several standard regression metrics on the test set: MAE (Mean Absolute Error): Indicates the average absolute deviation between predicted and true PTS values. MSE (Mean Squared Error): Penalizes larger errors more heavily. RMSE (Root Mean Squared Error): Interpretable in the same units as the target (PTS). R² Score: Measures how much of the variance in PTS the model explains.
The model achieved strong performance across all metrics, demonstrating that the relationship between the input features and the target variable (PTS) is highly predictable. To better visualize the model’s predictive accuracy, we plotted a True vs. Predicted Points graph. In this plot, each point represents a player-season example, comparing the actual PTS value with the model’s prediction. The diagonal 45-degree line represents perfect prediction; points lying close to this line indicate high predictive accuracy. The model’s predictions align closely with the diagonal, suggesting that the baseline linear regression effectively captures the underlying structure of the data.
Graph:
Feature Importance
To understand which attributes genuinely contribute to a player’s scoring output, a Linear Regression model was trained using only numerical features that do not directly determine the PTS value. This prevents data leakage from variables such as FGM, FTM, or 3PM, which mathematically construct the total points.
The feature importance shows several meaningful relationships:
- Turnovers (TOV) have one of the strongest positive coefficients, indicating that players who handle the ball more frequently tend to score more due to higher offensive usage.
- Games played (GP) has a strong negative coefficient, likely because consistent role players accumulate many games with lower scoring output, while high-volume scorers show greater variability.
- Physical factors such as height, weight, and personal fouls have moderate influence, suggesting that physical presence and playing style contribute indirectly to scoring.
- Other variables such as minutes, age, rebounds, blocks, steals, and assists show smaller yet still relevant effects.
This analysis provides an interpretable view of the factors influencing a player's scoring ability, focusing on genuine behavioral and physical attributes rather than variables that already encode point production.
Graph:
PCA Features
In addition to the original numerical and engineered features, I applied Principal Component Analysis (PCA) on the numerical feature space to extract a small number of latent components. Using three principal components allows the model to capture most of the variance in the numerical attributes while reducing dimensionality and noise. These PCA components are included alongside the original features in the preprocessing pipeline, providing the model with compact, information-rich representations of player performance.
Clustering-Based Feature
To capture higher-level player archetypes, I applied a K-Means clustering model on the scaled numerical features. The clustering was performed with five clusters, grouping players into distinct profiles based on their statistical and physical attributes. The resulting cluster index was added as a new categorical feature (cluster_kmeans) and later encoded in the preprocessing pipeline. This feature allows the regression model to leverage information about a player’s overall style or role (for example, high-usage scorers, defensive specialists, or low-usage role players), beyond what is captured by individual statistics alone.
Clustering Visualization (PCA Projection) To better understand the structure of the dataset and uncover natural groupings among players, a K-Means clustering model (with k = 5) was applied using a set of physical and defensive attributes: height, weight, rebounds, steals, blocks, assists, and turnovers. Since clustering operates without labels (unsupervised learning), each player is assigned to a numerical cluster ID (0–4), representing a statistically distinct group. To visualize these clusters, Principal Component Analysis (PCA) was used to project the multi-dimensional feature space into two components. The scatterplot below displays the players in this reduced 2D space, where each color represents a different K-Means cluster. The clear separation between clusters indicates that the model successfully identified meaningful player archetypes based on shared statistical patterns. These cluster assignments were then added back into the dataset as a new engineered feature (cluster_kmeans), enabling the regression models to leverage these latent player profiles for improved predictive performance.
Graph:
Cluster Interpretation
After applying K-Means clustering and visualizing the results using PCA, several clear patterns emerged that differentiate the player groups. Each cluster represents a distinct statistical profile based on core physical and defensive features (height, weight, rebounds, steals, blocks, assists, and turnovers). Although cluster IDs are numeric (0–4), the groups show meaningful structural differences:
Cluster 0 – Typically consists of lighter and shorter players with lower defensive metrics. These players tend to contribute less in rebounds and blocks, forming a “smaller guards” profile.
Cluster 1 – Characterized by players with slightly higher turnover rates and moderate height/weight values. This cluster appears to represent playmakers who handle the ball more frequently but may be less efficient defensively.
Cluster 2 – A large cluster of mid-sized players with balanced defensive contributions. They do not show extreme values in any category, suggesting a group of well-rounded but not highly specialized players.
Cluster 3 – Players with stronger defensive involvement (higher steals/blocks) combined with solid physical attributes. This group aligns with active defenders or hybrid wing players.
Cluster 4 – Represents the physically dominant group: taller and heavier players with the highest rebound numbers. This cluster clearly corresponds to traditional big men and centers.
These differences indicate that the clustering model successfully uncovered latent player archetypes that are not explicitly labeled in the dataset. Incorporating the cluster assignment as a new feature (cluster_kmeans) gives the regression model additional structural information, helping it better understand which “type” of player is being evaluated. This added context improves prediction accuracy and enhances the model’s ability to generalize.
Cluster-Based Feature Engineering Using the K-Means clustering model, each player was assigned a cluster label (cluster_kmeans), representing their statistical profile based on physical and defensive attributes.
To extract additional value from the clustering step, a second feature was created: dist_to_own_centroid — the distance between each player and the centroid of their assigned cluster. This captures how typical or atypical a player is within their group.
These cluster-derived features add meaningful latent structure to the dataset and help the regression model better differentiate between player types, improving predictive performance.
Model Performance Comparison
After building the baseline Linear Regression model and then applying feature engineering (including clustering-based features), three models were evaluated on the test set:
Baseline Linear Regression
- MAE: 59.67
- MSE: 8536.92
- RMSE: 92.40
- R²: 0.8899
Improved Linear Regression (with engineered features)
- MAE: 37.99
- MSE: 3788.13
- RMSE: 61.55
- R²: 0.9511
Random Forest Regressor
- MAE: 16.31
- MSE: 700.94
- RMSE: 26.48
- R²: 0.9910
Gradient Boosting Regressor
- MAE: 16.85
- MSE: 662.45
- RMSE: 25.74
- R²: 0.9915
The engineered features (including the cluster-based variables) significantly improved the Linear Regression model, reducing RMSE from ~92 to ~62 and increasing R² from 0.89 to 0.95. However, tree-based models performed substantially better than both linear baselines. Random Forest and Gradient Boosting achieved very low errors and R² values above 0.99, indicating that they capture almost all of the variance in the target variable (PTS).
Among all models, the Gradient Boosting Regressor achieved the best overall performance (lowest MSE/RMSE and highest R²), and is therefore selected as the final winner model for this project.
Feature Importance
To better understand what drives the predictions of the final Random Forest model, I examined the model’s feature importances after removing all leakage-prone features (such as minutes played, shooting percentages, and any direct transformations of PTS). The resulting plot shows that turnovers (TOV) and the distance to the player’s own cluster centroid are among the most influential predictors, followed by fouls (PF), steals (STL), games played (GP), BMI, age, and several defense-oriented metrics (def_impact, DRB, ORB, BLK). This indicates that the model relies mainly on a combination of usage style (how often a player is involved and loses the ball), physical profile, and defensive activity to estimate total points, rather than directly “cheating” with shooting or minutes-based statistics.
Graph:
Model Improvements – Discussion
The engineered models show a clear and substantial improvement over the baseline Linear Regression model.
After adding new features (per-minute ratios, usage proxy, defensive impact score, cluster ID, and distance-to-centroid), the Linear Regression already improved dramatically — raising the R² from 0.89 → 0.95 and cutting MAE by ~35%.
However, the largest gains came from switching to non-linear models.
Both Random Forest and Gradient Boosting achieved outstanding performance (R² ≈ 0.99), with error metrics less than half of the improved linear model.
The reasons for the improvement are:
- Non-linear relationships: Shooting volume, efficiency, and physical attributes interact in complex ways that linear models fail to fully capture. Tree-based models handle these interactions naturally.
- Feature engineering: Adding per-minute features, ratios, cluster embeddings, and distance-to-centroid provided structured, informative inputs that exposed deeper patterns in the data.
- Reduced leakage: By removing direct scoring-related variables (like FG/FT makes), the model learned true predictive signals rather than tautological relationships.
- Robustness of ensemble methods: Random Forest and Gradient Boosting reduce variance and can model heterogeneous player types far better than a simple regression.
Overall, the combination of engineered features + non-linear modeling produced a model that is significantly more accurate, generalizable, and interpretable than the baseline.
Winning Model
Based on all evaluation metrics (MAE, MSE, RMSE, R²), the clear winner is the Gradient Boosting Regressor.
It achieved the lowest overall error and the highest explanatory power, slightly outperforming the Random Forest model while maintaining excellent stability and generalization. The combination of non-linear modeling, robustness to feature interactions, and strong use of the engineered feature set makes Gradient Boosting the best-performing model in this project.
I chose to transform the continuous PTS target into three classes using quantile-based binning (bottom 33%, middle 33%, top 33%). This strategy creates roughly balanced class sizes, which is important for stable training and fair evaluation of classification models. It also provides a meaningful interpretation of the labels, distinguishing clearly between low-, medium-, and high-scoring players rather than using an arbitrary single threshold.
Class Balance Analysis:
After converting the regression target (PTS) into a binary classification target using a median split, we examined the class distribution in both the training and test sets. balanced dataset: approximately 50% of the samples belong to Class 0 (below median PTS) and 50% belong to Class 1 (at or above median PTS).
Because the classes are evenly distributed, there is no meaningful class imbalance. Therefore, standard evaluation metrics such as accuracy and F1 score are appropriate, and no resampling or class-weight adjustments are required.
Precision vs Recall
In the context of this task, the positive class represents high-scoring players (PTS at or above the median). From a scouting or decision-making perspective, it is more important to identify as many true high scorers as possible than to be extremely strict about avoiding false alarms. Therefore, recall for the positive class is more important than precision: we prefer to capture most of the truly high-scoring players, even if that means occasionally misclassifying some average players as high scorers.
False Positives vs False Negatives
A false negative in this setting means a truly high-scoring player is classified as a low scorer, which could lead to missing a valuable offensive asset. A false positive, on the other hand, means an average or low-scoring player is mistakenly treated as a high scorer, which can usually be corrected later through additional evaluation or data. For this reason, false negatives are more critical than false positives in this project, and the model should prioritize minimizing them (i.e., maximizing recall for the high-scoring class).
Classification Model Evaluation
For each classifier (Logistic Regression, Random Forest, and Gradient Boosting), I computed the classification report (precision, recall, F1-score) and plotted a confusion matrix using scikit-learn’s built-in tools. All three models achieved solid performance on the balanced binary classification task, but the tree-based models (Random Forest and Gradient Boosting) generally outperformed Logistic Regression in terms of F1-score and overall error rates.
From the confusion matrices, most predictions lie on the diagonal, indicating that the models correctly distinguish between low-scoring (class 0) and high-scoring (class 1) players in the majority of cases. The remaining errors are split between false positives (class 0 predicted as class 1) and false negatives (class 1 predicted as class 0). In the context of this project, false negatives are more critical because they correspond to truly high-scoring players being misclassified as low scorers, meaning the model fails to identify valuable offensive players. The better-performing models reduce the number of these false negatives, which aligns with the business goal of prioritizing recall for the high-scoring class.
Winning Classification Model
After comparing the performance of three classification models - Logistic Regression, Random Forest, and Gradient Boosting - the Gradient Boosting Classifier was selected as the final winning model.
It achieved the highest overall accuracy (97.6%), as well as the strongest balance between precision, recall, and F1-score across both classes.
Its confusion matrix shows that it makes the fewest misclassifications among all models, indicating superior generalization and stability.
Because of its consistently strong performance and minimal error rate, Gradient Boosting is declared the best-performing classifier, and was selected for export and deployment.