docs/models.md

Models Documentation

Overview

The system trains 22 models across two generations, then selects the best via unified evaluation. Champion v1: Random Forest (R² = 0.957, MAE = 4.78).
Champion v2.6: BestEnsemble — weighted average of RF + XGB + LGB, calibrated by R² score.

---

Model Versioning

Models are organized into two generations (v1 and v2) to support systematic ablation studies and deployment reproducibility. Ensemble methods are a patch release within v2.

Generation	Version range	Family	Description
v1	v1.0.0	Classical ML	Ridge, Lasso, ElasticNet, KNN ×3, SVR, RF, XGBoost, LightGBM
v2	v2.0–v2.5	Deep Learning	LSTM ×4, BatteryGPT, TFT, iTransformer ×3, VAE-LSTM
v2 patch	v2.6.0	Ensemble	BestEnsemble (weighted RF + XGB + LGB)

BestEnsemble (v2.6.0)

The weighted-average ensemble combines the three best classical models:

$$\widehat{y}=\frac{w_{\text{RF}}\cdot {\widehat{y}}_{\text{RF}}+w_{\text{XGB}}\cdot {\widehat{y}}_{\text{XGB}}+w_{\text{LGB}}\cdot {\widehat{y}}_{\text{LGB}}}{w_{\text{RF}}+w_{\text{XGB}}+w_{\text{LGB}}}\backslash hat\{y\}\; =\; \backslash frac\{w\_\{\backslash text\{RF\}\}\; \backslash cdot\; \backslash hat\{y\}\_\{\backslash text\{RF\}\}\; +\; w\_\{\backslash text\{XGB\}\}\; \backslash cdot\; \backslash hat\{y\}\_\{\backslash text\{XGB\}\}\; +\; w\_\{\backslash text\{LGB\}\}\; \backslash cdot\; \backslash hat\{y\}\_\{\backslash text\{LGB\}\}\}\{w\_\{\backslash text\{RF\}\}\; +\; w\_\{\backslash text\{XGB\}\}\; +\; w\_\{\backslash text\{LGB\}\}\}$$

Component	R²	Weight
Random Forest	0.957	0.957
XGBoost	0.928	0.928
LightGBM	0.928	0.928

The ensemble is registered automatically when all three components are loaded. See POST /api/predict/ensemble to use it directly.

---

Results Summary

Rank	Model	R²	MAE	RMSE	Family
1	Random Forest	0.957	4.78	6.46	Classical
2	LightGBM	0.928	5.53	8.33	Classical
3	Weighted Avg Ensemble	0.886	3.89	6.47	Ensemble
4	TFT	0.881	3.93	6.62	Transformer
5	Stacking Ensemble	0.863	4.91	7.10	Ensemble
6	XGBoost	0.847	8.06	12.14	Classical
7	SVR	0.805	7.56	13.71	Classical
8	VAE-LSTM	0.730	7.82	9.98	Generative
9	KNN-10	0.724	11.67	16.30	Classical
10	DG-iTransformer	0.595	9.38	12.22	Graph-Transformer
11	iTransformer	0.551	11.10	12.87	Transformer
12	BatteryGPT	0.508	10.71	13.47	Transformer
13	Vanilla LSTM	0.507	11.44	13.48	LSTM

---

1. Classical Machine Learning

1.1 Linear Models

Model	Regularization	Key Hyperparameters
Ridge	L2	α (cross-validated)
Lasso	L1	α (cross-validated)
ElasticNet	L1 + L2	α, l1_ratio

1.2 Instance-Based

• KNN (k=3, 5, 7): Distance-weighted, Minkowski metric

1.3 Kernel

• SVR (RBF): C, γ, ε via grid search

1.4 Tree Ensembles

• Random Forest: 500 trees, max_depth=None
• XGBoost: 100 Optuna trials, objective=reg:squarederror
• LightGBM: 100 Optuna trials, metric=MAE

All classical models use 5-fold battery-grouped CV for validation.

---

2. Deep Learning — LSTM/GRU Family

Built with PyTorch. Input: sliding windows of 32 cycles × 12 features.

2.1 Vanilla LSTM

• 2 layers, hidden_dim=128, dropout=0.2
• MAE loss, Adam optimizer

2.2 Bidirectional LSTM

• Same as Vanilla but processes sequences in both directions
• Doubles hidden representation

2.3 GRU

• 2-layer GRU (fewer parameters than LSTM)
• Simpler gating mechanism (reset + update gates)

2.4 Attention LSTM

• 3-layer LSTM + Additive Attention mechanism
• Learns to weight important time steps
• Attention weights are interpretable

Training Protocol

• Optimizer: Adam (lr=1e-3)
• Scheduler: CosineAnnealingLR
• Early stopping: patience=20
• Gradient clipping: max_norm=1.0
• Uncertainty: MC Dropout (50 forward passes, p=0.2)

---

3. Transformer Architectures

3.1 BatteryGPT

• Nano GPT-style decoder-only Transformer
• d_model=64, nhead=4, 2 layers
• Positional encoding + causal mask
• Lightweight (~50K parameters)

3.2 Temporal Fusion Transformer (TFT)

• Variable Selection Network for feature importance
• Gated Residual Networks for non-linear processing
• Multi-head attention with interpretable weights
• Originally designed for multi-horizon forecasting

3.3 iTransformer (Inverted)

• Inverts the attention axis: attends across features, not time
• Feature-wise multi-head attention + temporal convolution
• Built with TensorFlow/Keras

3.4 Physics-Informed iTransformer

• Dual-head: primary SOH head + auxiliary physics head (ΔQ prediction)
• Joint loss: L = L_soh + λ × L_physics (λ=0.3)
• Physics constraint regularizes learning

3.5 Dynamic-Graph iTransformer

• Adds Dynamic Graph Convolution layer
• Learns inter-feature adjacency matrix dynamically
• Fuses local (graph) and global (attention) representations

---

4. VAE-LSTM

• Encoder: 2-layer Bi-LSTM → μ, log σ² (latent_dim=16)
• Reparameterization: z = μ + σ · ε
• Decoder: 2-layer LSTM → reconstructed sequences
• Health Head: MLP(z) → SOH
• Loss: L_recon + β · KL + L_soh (β annealing over 30 epochs)
• Anomaly Detection: 3σ threshold on reconstruction error

---

5. Ensemble Methods

5.1 Stacking Ensemble

• Base models generate out-of-fold predictions
• Ridge regression as meta-learner
• Combines diverse model predictions

5.2 Weighted Average Ensemble (v2.6.0)

• Optimizes weights via L-BFGS-B (minimize MAE)
• Constraint: weights sum to 1, all ≥ 0
• Usually achieves best overall performance
• Registered as a v2 patch — no separate generation needed

---

Evaluation Metrics

Metric	Formula	Interpretation
MAE	mean(|y - ŷ|)	Average absolute error
MSE	mean((y - ŷ)²)	Penalizes large errors
RMSE	√MSE	Same units as target
R²	1 - SS_res/SS_tot	Explained variance (1.0 = perfect)
MAPE	mean(|y - ŷ|/y) × 100	Percentage error
Tolerance Accuracy	fraction within ±2%	Practical precision

---

6. Vectorized Simulation (predict_array)

Overview

The ModelRegistry.predict_array(X: np.ndarray, model_name: str) -> np.ndarray method enables batch prediction for the simulation pipeline without Python-level loops.

• Input: X — shape (N, n_features) where N is the number of simulation steps
• Output: flat np.ndarray of shape (N,) — SOH predictions for each step
• Automatically loads and applies the correct scaler via _load_scaler(model_name)
• Dispatches to the correct backend (sklearn .predict(), XGBoost/LightGBM .predict(), PyTorch .forward() batch, Keras .predict())

Simulation Pipeline (api/routers/simulate.py)

Each simulated battery follows this vectorized path:

1. Vectorized feature matrix assembled all at once using np.arange for cycle indices, scalar broadcasting for temperature/current/cutoff
2. All engineered features (SOC, cycle_norm, temp_norm, Δfeatures) computed column-by-column using numpy — no step loop
3. predict_array(X, model_name) called once per battery \u2192 entire SOH trajectory in one forward pass
4. RUL computed via np.searchsorted on the reversed-SOH array with the EOL threshold \u2192 O(log N) rather than O(N)
5. Degradation state classified by SOH thresholds using np.select([soh > 0.9, soh > 0.8, soh > 0.7], [...])

Physics Fallback (Arrhenius)

When no ML model is selected, pure physics degradation uses Arrhenius kinetics:

$$Q_{\text{loss}}=A\cdot \exp \text{\hspace{0.17em}⁣}(-\frac{E_a}{R\cdot T})\cdot N^{z}Q\_\{\backslash text\{loss\}\}\; =\; A\; \backslash cdot\; \backslash exp\backslash !\backslash left(-\backslash frac\{E\_a\}\{R\; \backslash cdot\; T\}\backslash right)\; \backslash cdot\; N^z$$

where $A = 31630$, $E_a = 17126\ \text{J/mol}$, $R = 8.314\ \text{J/(mol·K)}$, $z = 0.55$, and $T$ is temperature in Kelvin.

Performance

Vectorization replaces an O(N·k) Python loop (N steps × k overhead) with:

• Feature assembly: one np.column_stack call
• Prediction: single framework forward pass
• RUL: np.searchsorted O(log N)

For a 1 000-cycle simulation of 10 batteries this is 10–50× faster than the loop-based equivalent.