Neperl/diabetes-health-indicators-study

Diabetes Health Indicators, Model Comparison Study

This repository contains two machine learning models trained on the BRFSS 2015 Diabetes Health Indicators Dataset (balanced 50/50 split of 70,692 rows).

Task

Binary classification: Predict whether an individual has diabetes or prediabetes (Diabetes_binary = 1) versus no diabetes (Diabetes_binary = 0) based on 21 health indicators.

Models Included

1. Tuned Random Forest: Regularized tree ensemble (n_estimators=200, max_depth=10, min_samples_leaf=2).
2. Tuned MLP: Feedforward neural network with 1 hidden layer of 128 units, dropout=0.4, and Adam optimizer.

Final Performance Comparison

Model	Accuracy	Precision	Recall	F1-Score
Random Forest (Baseline)	0.7331	0.7176	0.7687	0.7423
Random Forest (Tuned)	0.7488	0.7281	0.7942	0.7597
MLP (Baseline)	0.7435	0.7179	0.8024	0.7578
MLP (Tuned)	0.7512	0.7264	0.8059	0.7641

Feature Importance (Random Forest Gini Importance)

The top indicators identified by the Random Forest model:

1. BMI (Body Mass Index)
2. Age (Age category 1-13)
3. GenHlth (Self-reported general health rating 1-5)
4. Income (Household income scale 1-8)
5. HighBP (High blood pressure indicator)

Repository Files

This repository contains the following files:

• diabetes_mlp_tuned.keras: The tuned Multilayer Perceptron (MLP) model.
• diabetes_random_forest_tuned.joblib: The tuned Random Forest classifier.
• scaler.joblib: The fitted StandardScaler used to normalize the feature inputs.
• diabetes_case_study.ipynb: The Jupyter notebook detailing the data download, exploratory analysis, preprocessing, training, tuning, and evaluation.
• README.md: Model card and description.

How to Load and Use

To make predictions on new data, you must first preprocess and scale the features using the saved scaler.

Loading the Scaler:

import joblib
scaler = joblib.load('scaler.joblib')

Scikit-Learn Model (Random Forest):

import joblib

# Load model
rf_model = joblib.load('diabetes_random_forest_tuned.joblib')

# Preprocess and predict
# X_scaled = scaler.transform(X_raw)
# y_pred = rf_model.predict(X_scaled)

Keras MLP Model:

from tensorflow import keras
import joblib

# Load model
mlp_model = keras.models.load_model('diabetes_mlp_tuned.keras')

# Preprocess and predict
# X_scaled = scaler.transform(X_raw)
# y_pred_prob = mlp_model.predict(X_scaled)

Conclusion

The tuned MLP marginally outperformed the tuned Random Forest in accuracy, recall, and F1 score. The neural network achieved the highest recall at 80.59%, which is critical in healthcare settings to limit false negatives. However, the MLP requires significantly more training time and architecture tuning than the ensemble model. Important risk factors identified by both methods include BMI, age, and general health.

Our finding that the tuned Random Forest and tuned MLP perform similarly aligns with broader research on tabular machine learning. Grinsztajn et al. (2022) benchmarked tree models against deep learning across 45 tabular datasets, showing that tree models remain superior or competitive on medium-sized tabular data. Their study found that neural networks struggle with the irregular target functions typical of tabular data and are more sensitive to uninformative features. This explains why hyperparameter tuning did not give the MLP a clear advantage over the Random Forest in this case study.

Limitations & Future Work

A main limitation is that the dataset relies on self-reported survey responses, which can introduce bias. The target column also groups prediabetes and diabetes into one class. Future research could explore gradient boosting models such as XGBoost or incorporate more complex feature engineering.

References

• L. Grinsztajn, E. Oyallon, and G. Varoquaux, "Why do tree-based models still outperform deep learning on typical tabular data?" Advances in Neural Information Processing Systems, vol. 35, pp. 507-520, 2022.