IEEE-CIS Fraud Risk Intelligence: LightGBM fraud ranking model
Model card for the scoring model of the IEEE-CIS Fraud Risk Intelligence project. This repository holds the card and the model registry record (model_registry.json); the trained weights are not uploaded here.
- Code and docs: GitHub
- Notebook: Kaggle
- Live dashboard: fraud-project-web.vercel.app
- Case study: portfolio
Model details
| Item | Value |
|---|---|
| Algorithm | LightGBM binary classifier |
| Model version (registry) | lightgbm-v2-v339-missingness-filtered |
| Validation strategy (registry) | time-based holdout plus expanding rolling cross-validation |
| Feature count | 425 features, 26 of them categorical |
| Scoring script parameters | learning_rate=0.035, num_leaves=64, subsample=0.85, colsample_bytree=0.85, class_weight="balanced", random_state=42 |
| Default number of estimators | 500 |
Anonymous Vesta V features are included when train missingness is at or below the configured ceiling (0.95 in the registry).
Training data
The model is trained on the IEEE-CIS Fraud Detection data: 590,540 transactions, $79.7M, 3.50% baseline fraud rate. The project README counts 20,663 fraud-labeled train transactions. The dataset page says the TransactionDT feature is a timedelta from a given reference datetime (not an actual timestamp).
Validation
The validation split is time-based: transactions are ordered by TransactionDT, and the last 20% of the training window is used as the validation holdout.
| Metric (time-based holdout) | Value |
|---|---|
| Validation rows | 118,108 |
| Validation fraud baseline | 3.44% |
| ROC-AUC | 0.9134 |
| Average precision | 0.5354 |
| Rolling CV mean ROC-AUC (three expanding windows) | 0.9100 |
The top 5% validation score band is the focused control scenario: 40.13% precision, 58.32% recall, and 5.00% workload.
The Kaggle notebook, as summarised in the project README, reports that on the same model random 5-fold gives ROC-AUC 0.971 and the time split 0.921; about half the gap is customers seen in training, half is time drift.
Limitations and intended use
- The model is not calibrated: balanced class weights inflate the score 5.2x against the observed rate, and the expected calibration error is 14.52%.
- The score ranks well and should be used only as a rank.
- It should not be used as an autonomous decline engine without calibrated probabilities, production decision logs, model-risk approval, and bank-specific cost validation.
Reproduce
The training and scoring code is src/prepare_raw_and_ml.py in the GitHub repository; the registry record in this repository is a copy of docs/model_registry.json from the same repository.
License
MIT License, Copyright (c) 2026 Senanur Cetin (same as the GitHub repository).