navk8690/paymind-reference-models-v2

PayMind Reference Models — V4

Multi-model reference artifacts for the PayMind open-source payment intelligence connector.
Models predict. PayMind decides.

PayMind evaluates eligible payment routes using route-fit, reliability, settlement-time and fee signals, then passes those signals to a configurable ranking policy. This repository contains the frozen V4 reference model baseline.

CatBoost · LightGBM · XGBoost   |   200k synthetic training transactions   |   Canonical Benchmark v1: 50k synthetic transactions   |   GPL-3.0

Important: These are synthetic reference models for development, demonstration and benchmarking. They are not production-calibrated provider models and must not be used to compare the real-world performance of payment providers.

At a glance

Engine	Question answered	ML task	Output
Candidate Generator	Which routes fit this transaction?	Multiclass classification	Route relevance probabilities
Reliability Engine	How likely is this route to succeed?	Binary probability prediction	P(success)
Settlement Intelligence	How quickly should it settle?	Quantile regression	P50 / P90 arrival time
Fee Engine	What will this route cost?	Deterministic logic	Estimated fee
Ranking Engine	Which eligible route should rank highest?	Configurable policy	Final route ranking

The first three responsibilities use ML. Fee and ranking are deliberately kept outside the predictive model layer.

V4 reference results

Engine	Model	Metric	V4 result
Candidate	CatBoost	Top-1 accuracy ↑	19.02%
Candidate	CatBoost	Top-3 accuracy ↑	48.33%
Reliability	CatBoost	ROC-AUC ↑	0.6480
Reliability	LightGBM	ROC-AUC ↑	0.6437
Reliability	XGBoost	ROC-AUC ↑	0.6419
Settlement	CatBoost	P50 MAE ↓	13.02 min
Settlement	CatBoost	P90 coverage ≈ 90%	93.51%

↑ higher is better · ↓ lower is better · coverage is a calibration target, not a “higher is always better” metric.

These results are measurements inside the PayMind V4 synthetic environment only.

Architecture

Transaction
    │
    ▼
Eligibility Engine
    │
    ▼
Candidate Generator ─────────────── route-fit
    │
    ├───────────────┐
    ▼               ▼
Reliability      Settlement
Engine           Intelligence
    │               │
    └───────┬───────┘
            ▼
        Fee Engine
            │
            ▼
       Ranking Engine
            │
            ▼
    Ranked eligible routes

Each predictive responsibility can be evaluated by multiple gradient-boosting families:

Model family	Why it is included
CatBoost	Strong fit for mixed tabular/categorical payment features; current V4 reference leader on reliability.
LightGBM	Efficient gradient boosting with an independent learning implementation; useful for comparison and ensemble diversity.
XGBoost	Mature gradient-boosted tree framework providing a third independent predictive view.

The objective is not to assume that three models are automatically better than one. PayMind can benchmark their standalone performance and test whether their errors are complementary before assigning ensemble weights.

1. Candidate Generator

The Candidate Generator estimates route relevance for a transaction. It is a multiclass classification problem: given transaction context, the model produces a probability distribution across candidate route classes.

Example:

Route A   0.31
Route B   0.24
Route C   0.18
Route D   0.11

The highest candidate probability is not the final PayMind decision. Reliability, settlement, fees, eligibility and ranking policy still influence the recommendation.

Candidate metrics

Metric	Meaning
Top-1 ↑	Target route is the model's highest-ranked candidate.
Top-3 ↑	Target route appears among the three highest-ranked candidates.
Log loss ↓	Quality of the full predicted probability distribution.

V4 CatBoost: Top-1 19.02% · Top-3 48.33%.

Top-3 is particularly useful in PayMind because candidate generation is an upstream shortlist signal rather than the final routing decision.

2. Reliability Engine

The Reliability Engine estimates:

How likely is this transaction to succeed through this candidate route?

For each transaction-route pair it produces a probability such as:

P(success) = 0.87

Route relevance and reliability are intentionally separate: a route may fit a transaction well while another eligible route has a stronger predicted success probability.

Reliability metrics

Metric	Interpretation
ROC-AUC ↑	Ability to discriminate between successful and unsuccessful outcomes. 0.50 is approximately random; 1.00 is perfect discrimination.
PR-AUC ↑	Precision-recall performance; useful when outcome classes are imbalanced.
Brier ↓	Squared error of predicted probabilities.
Log loss ↓	Penalizes poor probability estimates, especially confident errors.

V4 reliability comparison

Model	ROC-AUC ↑
CatBoost	0.6480
LightGBM	0.6437
XGBoost	0.6419

The three results are close enough to make ensemble experiments worthwhile. The best ensemble should be selected from fixed-benchmark evidence, not simply by averaging models or choosing weights arbitrarily.

3. Settlement Intelligence

Settlement Intelligence estimates both typical arrival time and slower-tail risk:

• P50 — approximately the median expected settlement time.
• P90 — a more conservative estimate intended to capture slower outcomes.

For example:

Route	P50	P90
A	2 min	4 min
B	2 min	3 hr

Both routes look equally fast at the median, but Route B has substantially greater tail risk. That distinction matters when settlement speed is operationally important.

Settlement metrics

Metric	Interpretation
P50 MAE ↓	Average absolute error of median settlement predictions.
P50 coverage ≈ 50%	Calibration target for the median quantile.
P90 MAE ↓	Average absolute error of P90 predictions.
P90 coverage ≈ 90%	Calibration target for the conservative quantile.

V4 CatBoost: P50 MAE 13.02 minutes · P90 coverage 93.51%.

A P90 coverage of 99% is not automatically better than 90%; it may indicate an overly conservative model. Coverage is judged by proximity to the target quantile.

Multi-model inference

PayMind V4 supports CatBoost, LightGBM and XGBoost as independent predictive families. A future weighted ensemble can conceptually use:

ensemble =
    w_cb   × CatBoost
  + w_lgbm × LightGBM
  + w_xgb  × XGBoost

where w_cb + w_lgbm + w_xgb = 1

The important rule is that ensemble weights should be learned or selected against the fixed benchmark. A model can add ensemble value even when its standalone metric is slightly lower if it makes different errors from the other members.

Predictive models vs decision policy

PayMind intentionally separates prediction from business preference.

Predictive layer	Decision layer
Route relevance	Eligibility rules
Success probability	Fee policy
P50/P90 settlement	Urgency preference
Model ensemble	Industry/business profile
Probability calibration	Final ranking weights

This means a trading-oriented deployment can prioritize settlement speed more aggressively than a cost-sensitive commerce deployment without retraining the predictive models solely to encode that preference.

V4 benchmark

Property	V4 baseline
Synthetic environment	V4
Training scale	200,000 transactions
Evaluation set	Canonical Benchmark v1
Benchmark scale	50,000 transactions
Model families	CatBoost / LightGBM / XGBoost
Status	Frozen reference baseline

The benchmark is separate from training observations and is intended to remain fixed while model tuning, calibration and ensemble experiments are compared. Changing both the model and benchmark at the same time makes improvement harder to attribute.

Synthetic environment

The V4 environment includes synthetic variation across transaction type, amount bands, currencies, geography, domestic/cross-border context, time/day context, route suitability, success/failure behaviour, and instant versus slower settlement behaviour.

V4 intentionally contains meaningful instant and near-instant settlement behaviour while retaining structured slower tails associated with synthetic contexts such as cross-border activity, withdrawals, high-value transactions, weekends, banking hours and bank-transfer-style routes.

These are generated behavioural relationships, not observations about real payment providers.

Repository layout

registry.json
registry.multimodel.json
payment_method/
  payment_method_v1.cbm
  metadata.json
success/
  success_v1.cbm
  metadata.json
arrival/
  arrival_p50_v1.cbm
  arrival_p90_v1.cbm
  metadata.json
multimodel/
  lightgbm/
  xgboost/

CatBoost artifacts use .cbm. LightGBM and XGBoost artifacts use the formats configured by the PayMind model registry.

Using the models

These artifacts are designed to be loaded by PayMind's Model Registry, rather than as generic Hugging Face Transformer models.

from paymind import PayMind

engine = PayMind()
result = engine.evaluate({
    # canonical PayMind transaction request
})

At a high level:

request → features → registry → model inference
        → eligibility/fees → ranking → recommendation

Intended use

These models are suitable for PayMind development, demonstrations, architecture testing, benchmarking, multi-model experiments, ensemble research, settlement-aware ranking experiments and educational use.

They should not be treated as production-calibrated gateway models, verified provider benchmarks, real success-rate or settlement-SLA estimates, real fee estimates, financial advice, or a payment execution system.

Production use

Before real-world deployment, retrain, calibrate and validate PayMind using data representative of the target payment environment. Production quality will depend on data quality, feature availability, transaction distribution, geography, payment methods, provider behaviour and operational changes over time.

A typical path is:

own transaction data
→ schema validation
→ cleaning / features
→ chronological splits
→ model training
→ fixed validation benchmark
→ calibration
→ policy validation
→ controlled deployment

Limitations

• Synthetic data: V4 learns generated relationships rather than observed production-provider behaviour.
• Distribution shift: a real payment environment may differ materially from the reference environment.
• Calibration: probabilities and quantiles must be revalidated on target data.
• Provider change: real payment methods, availability, commercial terms and processing behaviour evolve.
• Ranking is separate: better predictive metrics do not automatically imply a better business routing policy.
• Benchmark scope: Canonical Benchmark v1 is a PayMind synthetic benchmark, not an industry benchmark.

Privacy and safety

The public reference repository is intended to contain model artifacts and metadata only. It should not contain production transaction CSVs, private training/processed data, credentials, API keys, environment secrets or private model artifacts.

Provider names appearing in synthetic examples or classes do not imply access to proprietary provider data or establish real provider performance.

License

PayMind reference model artifacts are released under the GNU General Public License v3.0 (GPL-3.0), subject to the repository's license terms.

---

PayMind

PayMind is an open-source payment intelligence connector — not a payment gateway.

These V4 models are synthetic reference artifacts. They do not execute payments, move money, hold funds, or provide verified predictions about the real-world performance of any payment provider.