Arko007/trustworthy-gnn-fraud-models

<model_card

Trustworthy GNN Fraud Detection Models

Graph Neural Network models for illicit transaction detection on the Elliptic Bitcoin Dataset, with conformal prediction calibration and evidential deep learning uncertainty quantification.

Model Architecture

Each model uses residual connections (skip connections with linear projection when dimensions differ), LayerNorm, and DropEdge regularization. All models are trained with Focal Loss (γ=2) with label smoothing (ε=0.05) and a warmup + cosine annealing schedule.

Backbones

Backbone	Conv Layer	Activation	Normalization	Heads
GraphSAGE	SAGEConv	ReLU	LayerNorm	—
GAT	GATConv	ELU	LayerNorm	4
GCN	GCNConv	ReLU	LayerNorm	—

Topologies

Topology	Description	Edges
original	Raw transaction graph directly from Elliptic dataset
temporal	Nodes linked across consecutive timesteps via k-NN (k=5) in feature space
knn	k-nearest neighbors graph (k=10) based on feature similarity
similarity	Cosine similarity graph (threshold=0.92)
augmented	Original edges + temporal edges combined

Feature Engineering

All models use 499-dimensional features:

• 166 original features (RobustScaler normalized)
• 3 degree features (log total/in/out degree)
• 1 PageRank score (log-transformed, α=0.85)
• 1 Clustering coefficient
• 328 neighbor aggregation stats (mean + std of neighbor features for each original dimension)

Metrics

Best Model: graphsage_temporal_elliptic

This model achieved the highest AUC-ROC across all configurations.

AUC-ROC:        0.7417
Coverage rate:  ~0.91

All Models (sorted by AUC-ROC)

Model	AUC-ROC
graphsage_temporal	0.7417
graphsage_augmented	0.7065
graphsage_original	0.6938
gcn_original	0.6796
gat_original	0.6796
gat_similarity	0.6727
ensemble (n=15)	0.6743
gcn_augmented	0.6661
gat_augmented	0.6637
graphsage_knn	0.6385
gcn_knn	0.6214
gat_knn	0.6214
gcn_similarity	0.6211
graphsage_similarity	0.6131
gat_temporal	0.5957
gcn_temporal	0.5122

Note: The direct average ensemble of all 15 models underperforms individual best models. A weighted or selective ensemble (e.g., top-5) would likely perform better.

EDL (Evidential Deep Learning) Models

Three EDL models are available with 166-dim features (original features only, no engineering):

EDL models output Dirichlet concentration parameters (α) instead of logits, enabling uncertainty decomposition into aleatoric and epistemic components.

Conformal Calibration

Every model is calibrated using Adaptive Prediction Sets (APS) scoring with Mondrian conformal prediction:

• Alpha: 0.1 (target 90% coverage)
• Scoring function: APS (sorted probability + uniform random tiebreak)
• Calibration set: Validation split (timesteps 35–42)
• Coverage rate: ~0.91–0.92 on test set

Calibration thresholds are stored in conformal_calibration.json per model, with global and class-conditional (Mondrian) quantile thresholds.

Training Details

• Hardware: Kaggle CPU only (Intel Xeon, 4 vCPUs)
• Training time: ~9 hours for all 15 models + ensemble
• Epochs: 300 (early stopping patience=30)
• Optimizer: AdamW (lr=3e-4, weight_decay=1e-4)
• Scheduler: Linear warmup (10 epochs) + CosineAnnealingLR
• Loss: Focal Loss (γ=2, label smoothing=0.05)
• Batch: Full-batch gradient descent (transductive GNN)
• DropEdge: 0.1 edge dropout rate during training

Hyperparameters

Parameter	Value
Hidden dimension	256
Number of layers	3–4
Dropout	0.25
Learning rate	3e-4
Weight decay	1e-4
Focal γ	2.0
Label smoothing ε	0.05
Gradient clipping	3.0
DropEdge rate	0.1

Usage

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from backbones import GraphSAGEBackbone  # provides LayerNorm + residuals

model_name = "graphsage_temporal_elliptic"
path = hf_hub_download(
    repo_id="Arko007/trustworthy-gnn-fraud-models",
    filename=f"{model_name}.safetensors"
)
state_dict = load_file(path)

model = GraphSAGEBackbone(in_channels=499, hidden_channels=256, out_channels=2, num_layers=3, dropout=0.25)
model.load_state_dict(state_dict)
model.eval()

Loading with the backend API

from models.loader import ModelLoader
loader = ModelLoader()
model = loader.load_model("graphsage_temporal_elliptic")

License

MIT

References

• Weber et al., "Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics", KDD 2019 AML Workshop
• Angelov et al., "Evidential Deep Learning for Trustworthy Prediction", 2022
• Angelopoulos & Bates, "Conformal Prediction: A Gentle Introduction", Foundations and Trends in Machine Learning, 2023
• Elliptic Data Set: https://www.kaggle.com/datasets/ellipticco/elliptic-data-set