🍃 KaoLEAF

An attention-based credit-decision model that explains itself.
Calibrated 300–850 score · five A–E risk bands · additive reason codes a credit officer can read.

Teerapong Panboonyuen · Government Savings Bank Artificial Intelligence Laboratory (GSBAIL)

Status. The manuscript is in preparation, so numbers and wording may change before publication. This repository releases the trained weights and a reproducible reference implementation. It is a research artifact, not a lending system.

---

✨ Why KaoLEAF

Most tabular credit models give you a number. KaoLEAF gives you a number you can defend.

🧭 Knowledge-ordered tokens	24 engineered features become tokens arranged by financial meaning: profile → credit behaviour → capacity → request.
🔢 Learned periodic embeddings	Numeric features get a learned periodic embedding instead of a raw scalar.
🧩 Gated depthwise-separable conv + multi-head self-attention	Local mixing inside a financial block, global interactions across blocks.
🪜 Two-stage learning	Self-supervised masked-feature modelling, then supervised fine-tuning, repeated over a 5-seed deep ensemble.
🎯 Calibrated by design	Temperature scaling on validation (T = 1.044), a 300–850 score, five monotone bands.
🔍 Auditable	Integrated Gradients with a completeness check, attention rollout, a SHAP cross-check, ICE/PDP monotonicity tests, slice/fairness audit.
🧱 Layered	Representation, outcome prediction and decision policy are separate, so thresholds and hard rules change without retraining.

 applicant record (18 raw fields → 24 engineered features)
        │
        ▼
 ┌─────────────────────────────────────────────────────────────┐
 │ Feature tokeniser   periodic embedding (numeric, k=8, σ=0.5)│
 │                     learned embedding (categorical)         │
 │ Knowledge order     Profile → Credit behaviour → Capacity → │
 │                     Request                                 │
 │ Local operator      gated depthwise-separable 1-D conv (k=3)│
 │ Global operator     3 × multi-head self-attention (d=64, h=4)│
 │ Behaviour embedding z  ──►  logistic head                   │
 └─────────────────────────────────────────────────────────────┘
        │  × 5 seeds, mean logit ÷ T (temperature scaling)
        ▼
 P(approve)  ──►  KaoLEAF Score = round(300 + 550·p)  ──►  Band A–E  ──►  Reason codes (IG)

---

📊 Results

Benchmark: Loan Approval (US & Canada), 50,000 applicants, stratified 70/10/20 split (seed 42), 10,000 held-out test applicants. Every model receives the same features, and the tree baselines were tuned and early-stopped on the same validation set.

Model	Test AUC	PR-AUC	KS	ECE ↓	Brier ↓
KaoLEAF-Fusion (KaoLEAF + GBDT stack)	0.9852	0.9881	0.8617	0.0096	0.0484
LightGBM (Optuna)	0.9850	0.9879	0.8607	0.0108	0.0487
CatBoost	0.9849	0.9879	0.8607	0.0129	0.0489
XGBoost	0.9842	0.9874	0.8581	0.0137	0.0501
HistGradientBoosting	0.9835	0.9866	0.8533	0.0111	0.0511
⭐ KaoLEAF (5-seed ensemble, calibrated) — the weights in this repo	0.9825	0.9858	0.8478	0.0107	0.0529
FT-Transformer (plain)	0.9798	0.9837	0.8439	0.0108	0.0564
MLP (embeddings)	0.9744	0.9794	0.8199	0.0106	0.0638
Random Forest	0.9737	0.9789	0.8216	0.0561	0.0683
Extra Trees	0.9667	0.9740	0.7964	0.0953	0.0848
Logistic Regression	0.9427	0.9511	0.7289	0.0080	0.0947

🧐 How to read this honestly

• The weights in this repo are the standalone KaoLEAF ensemble (AUC 0.9825). The Fusion row stacks KaoLEAF with gradient-boosted trees; those tree models are not part of this release.
• KaoLEAF-Fusion ranks first, but its margin over tuned LightGBM is small (+0.0002 AUC), and the 95% bootstrap intervals of the top models overlap.
• The standalone ensemble is competitive but sits behind the tuned gradient-boosted trees. It beats the plain FT-Transformer (+0.0027 AUC), the MLP (+0.0081) and logistic regression (+0.0397).
• In the single-seed ablation, only the periodic embedding shows a consistent gap. Other differences are inside seed noise.
• The case for KaoLEAF is ranking that is competitive with boosting, plus a reusable embedding, calibrated scores, stable bands and faithful explanations. It is not a claim of beating boosting on AUC.

🎚️ Score bands (test set)

Band	Score range	Applicants	Share	Mean predicted	Observed approval
A · Very strong	740–850	4,632	46.3%	97.5%	97.8%
B · Strong	631–740	609	6.1%	70.4%	71.1%
C · Moderate	520–630	464	4.6%	50.3%	57.8%
D · Weak	410–520	504	5.0%	29.7%	33.9%
E · Very weak	300–410	3,791	37.9%	2.3%	2.7%

Observed approval rises monotonically from E to A and tracks the predicted rate in every band.

---

🚀 Quick start

pip install torch numpy pandas scikit-learn scipy joblib huggingface_hub

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("kaopanboonyuen/KaoLEAF")   # weights + preprocessor + kaoleaf.py
sys.path.insert(0, path)
import kaoleaf as kl

scorer = kl.KaoLEAFScorer.load(path, device="cpu")

applicant = dict(
    age=29, occupation_status="Employed", years_employed=4.0, annual_income=52000,
    credit_score=655, credit_history_years=6.0, savings_assets=3500, current_debt=9000,
    defaults_on_file=0, delinquencies_last_2yrs=1, derogatory_marks=0,
    product_type="Personal Loan", loan_intent="Education", loan_amount=15000,
    interest_rate=15.5, debt_to_income_ratio=0.31, loan_to_income_ratio=0.29,
    payment_to_income_ratio=0.10,
)

res = scorer.predict(applicant)
print(res.p_approve[0], res.kaoleaf_score[0], res.risk_band[0])
# 0.9997  850  A - Very strong

why, _ = scorer.explain(applicant, top_k=4)       # Integrated Gradients reason codes
for feature, value, attribution in why[0]["reasons_against_approval"]:
    print(kl.format_reason(feature, value, attribution))

A runnable copy is in examples/quickstart.py. To try it without code, open the live demo.

Input schema

scorer.predict takes raw applicant fields; the 6 extra ratios are engineered inside kaoleaf.engineer.

Field	Type	Notes
age	int	Protected attribute, see Limitations
occupation_status	str	Employed · Self-Employed · Student
years_employed, credit_history_years	float	years
annual_income, savings_assets, current_debt, loan_amount	number	currency units as in the dataset
credit_score	int	300–850
defaults_on_file, delinquencies_last_2yrs, derogatory_marks	int	counts (defaults_on_file is 0/1)
product_type	str	Credit Card · Personal Loan · Line of Credit
loan_intent	str	Business · Debt Consolidation · Education · Home Improvement · Medical · Personal
interest_rate	float	percent
debt_to_income_ratio, loan_to_income_ratio, payment_to_income_ratio	float	in the dataset these are current_debt/income, loan_amount/income and ≈ loan_to_income/3

Keep decisions as policy, not as model weights

def decision_policy(p, defaults_on_file, approve=0.60, review=0.40):
    if defaults_on_file == 1:
        return "DECLINE (policy rule: default on file)"
    return "APPROVE" if p >= approve else "MANUAL REVIEW" if p >= review else "DECLINE"

In the data, defaults_on_file = 1 is never approved (0 of 2,674 applicants), so it is enforced as a rule rather than learned.

🔐 Security note. kaoleaf_prep.joblib is a pickle-based file (a fitted scikit-learn preprocessor). Hugging Face will flag it as pickle. Load it only from sources you trust, such as this repository at a pinned revision.

---

🔍 Explainability and audit

• Global importance from mean |Integrated Gradients|, per feature and per financial block.
• Cross-check against LightGBM SHAP: Spearman ρ = 0.72 between two model families and two attribution methods.
• Reason codes per applicant, in calibrated-logit units, in the structure of an adverse-action list.
• Attention rollout is reported as a descriptive diagnostic, not a causal explanation. The faithful additive explanation is Integrated Gradients.
• Monotonicity audit with ICE/PDP for credit score (↑), DTI (↓), delinquencies (↓) and credit history (↑).
• Slice audit by age group, occupation, product and loan intent, with the adverse-impact ratio as a screening statistic, not a legal verdict.

---

📦 What is in this repository

File	What it is
kaoleaf_weights.pt	5 ensemble members + temperature (≈3.9 MB)
kaoleaf_prep.joblib	fitted preprocessor (quantile transform, category levels, median-applicant baseline)
kaoleaf_config.json	architecture config, feature order, bands, dataset SHA-256, seed, stacker weights
kaoleaf.py	model, preprocessing, training, explainability and the KaoLEAFScorer API
metrics.json, leaderboard.csv	test metrics for every model in the benchmark
test_predictions.csv	score, band and probability for each of the 10,000 test applicants
run_manifest.json	library versions, seed, GPU
figures/	selected results, explainability and audit figures
examples/quickstart.py	download-and-score example

Integrity. Weights SHA-256: d8086702aaaa871a35187e61e7a4a71f21f1e49bc642c430088753cfce5f2560. Dataset zip SHA-256: aa6435d06ea46ec256b79a2780faee807ba63334df4c4ce507aa9bc2097ae61f.

Training recipe. Stage 1: masked-feature modelling, 25 epochs. Stage 2: supervised fine-tuning, up to 60 epochs with early stopping. Seed 42, ensemble members use seed + s. Runs used Python 3.13, PyTorch 2.11 (CUDA 13.0) and a Colab T4. The model also runs on CPU. GPU kernels can still move AUC by about 1e-4, so report mean ± std over seeds.

---

⚠️ Limitations and governance

• Cross-sectional data. One row per applicant. The local operator is a 1-D convolution over ordered financial features, not over time. Temporal modelling is not evaluated.
• Likely simulated data. The public dataset appears synthetic or realistic-simulated, so results show method behaviour, not real bank performance.
• Protected attribute. age is protected under US (ECOA) and Canadian lending rules. A variant trained without it is supported (DROP_FEATURES=["age"] in the notebook) and recommended for any deployment. The released weights include age.
• Possible leakage. Pricing variables (interest_rate, payment_to_income_ratio) may be outputs of underwriting in production.
• Single dataset, single split. Bootstrap intervals reflect test sampling only, not split variance.
• Not validated for production lending. No GSB customer data is used.

Out-of-scope use. Do not use these weights to make or support real credit decisions about real people.

---

🗺️ Roadmap

• Multi-seed, multi-split reporting for the paper
• Monotonic-constraint variant for regulatory use
• Knowledge distillation from gradient-boosted trees into KaoLEAF
• Longer Stage-1 pre-training on larger unlabeled pools
• Temporal encoder on longitudinal data (planned for GSB data, which cannot be public)

---

📝 Citation

The manuscript is in preparation. Until then, please cite the project:

@misc{panboonyuen2026kaoleaf,
  title  = {KaoLEAF: Knowledge-Augmented Optimization and Learned Embeddings
            for Automated Feature Representation},
  author = {Panboonyuen, Teerapong},
  year   = {2026},
  note   = {Government Savings Bank Artificial Intelligence Laboratory (GSBAIL).
            Manuscript in preparation.
            Code and project page: https://kaopanboonyuen.github.io/KaoLEAF}
}

🙏 Acknowledgements

All results were obtained on the Realistic Loan Approval Dataset | US & Canada by Parth Patel (parthpatel2130) on Kaggle. Please follow the license terms on the original Kaggle page. A split-preserving copy with full credit is at kaopanboonyuen/loan-approval-us-ca.

📄 License

The code and weights in this repository are released under the Apache-2.0 license. The training data is a third-party dataset under its own terms (see above).