jshmhsb/EKPT-backup

Forvera — Foundation model for Real-world Variable-time Event Representation and Adaptation

A research codebase for long-window irregular user-event modeling. Forvera builds a causal, vector-valued event field from marked search/browse/cart/purchase streams, integrates event-type-specific temporal kernels analytically over multi-resolution real-time patches, and carries long-window dependencies with a patch-level Transformer. The same encoder feeds any of three forecast paradigms.

The public model name is Forvera. The existing Python package ekpt, model class EKPT, configuration filenames, and repository paths remain compatibility identifiers until a separate code-migration decision is made.

Design reference: long_window_irregular_event_modeling_report.md.

Documentation map: docs/README.md.

Research continuity: session_log.md.

Experiment results: dashboard. The latest Moreva run is E043 — pretrained Mamba-3 backbone for all-as-mark TPP; its structured zero-shot evaluation, training history, and final checkpoint are stored beside the report.

Current MoreVA paper snapshot: moreva/README.md summarizes the motivation, event-native Mamba-3 interface, aggregate multi-event results, attribution controls, limitations, and the exact code and paper artifacts in this repository. The 187M E043 checkpoint above is an earlier prototype; it is not the 1.5B system evaluated in the current paper.

Paper artifacts: short.pdf, long.pdf, and the shared appendix.pdf. All publication-specific sources, drafts, and figures live under ekpt-paper/.

Architecture proposals: Hawkes kernels, Chronos-predicted latent behavior states, and interval prediction.

Pipeline

marked events
  -> event encoding (type / entity / content + robust marks, FiLM fusion)
  -> optional local causal relational block (short event chains)
  -> causal multi-scale kernel + analytic multi-resolution patches
  -> patch-level Transformer backbone (long dependencies)
  -> forecast head:  parametric | language-model | generative

Layout

Module	Purpose
ekpt.kernels	Causal exponential / multi-scale mixture / low-rank type-pair kernels with analytic interval integrals
ekpt.encoders	Event vector encoding and robust per-type mark encoders
ekpt.patches	Multi-resolution patch boundaries and analytic patch aggregation (0th/1st/2nd temporal moments)
ekpt.backbone	Patch Transformer with temporal position encoding; local causal relational attention with reliability gate
ekpt.heads	Parametric (point/quantile/Bernoulli), language-model (scalar bins), generative (flow matching) heads
ekpt.data	(time, entity, type, mark, context, mask, action/exposure) event contract with strict cutoff enforcement
ekpt.model	Composed Forvera model, currently exposed as class EKPT

Three forecast paradigms (Sundial taxonomy)

1. Parametric density / point regression — MSE, pinball, or NLL over continuous futures (default; Chronos-2 reference).
2. Language modeling — cross-entropy over discretized scalar/event/codebook tokens (Chronos v1 reference).
3. Generative modeling — flow matching over continuous future patches (Sundial reference).

Tokenization and forecast paradigm are independent axes; the model family is assigned from the predicted target and loss, not from whether the input contains discrete IDs.

Cutoff safety

The prediction-cutoff rule (design doc §6.4) is enforced in the data contract and verified in tests/test_cutoff_safety.py:

• Prefix consistency — the state at query time q is identical whether or not post-q events are loaded.
• Future-permutation invariance — permuting post-cutoff events cannot change the causal output.
• Serving parity — the same prefix produces the same state whether scored singly or in a padded batch.

Quickstart

Environments are managed with uv. On this Cloud Desktop, install the Python selected by .python-version, create .venv, and install the exact locked environment with:

uv python install
uv sync --all-extras

uv run --all-extras pytest tests/ -q
uv run --all-extras mypy src/ekpt
uv run --all-extras python scripts/smoke_test.py

pyproject.toml is the only direct-dependency declaration and uv.lock pins the complete transitive environment. CI and production should use uv sync --all-extras --frozen; add or remove packages with uv add / uv remove, never with pip install.

Status

Scaffold with working forward/backward passes, analytic-integral kernels verified against closed forms, and passing cutoff-safety tests. Not yet wired to real CBFM event data — see the design doc §8 for the minimum viable experiment plan.