admincybers2/TRM_model_3

TRM chunk selector

Recursive PASS/FAIL gate over retrieved chunks (TRM core + skip head), 1.91M params on frozen Azure text-embedding-ada-002 embeddings (dim 1539). Decides per chunk whether it belongs in the answer set — a variable-size selection instead of a fixed top-k.

Test metrics (threshold 0.961)

micro-P 0.8462 · micro-R 0.6962 · micro-F1 0.7639 · exact-set 0.2 · best epoch 46

bench	groups	P	R	F1
gold_easy	9	1.0	0.4138	0.5854
gold_medium	2	0.9714	0.8947	0.9315
gold_hard	5	0.9468	0.9889	0.9674
llm_held_out	29	0.5556	0.375	0.4478

gold_* tiers are hand-curated deterministic labels (easy = section how-to, medium = single-doc, hard = table/matrix incl. reverse lookups); rephrasing variants of those questions are in train, so they measure learned question types. llm_held_out is strict generalization on unseen questions.

Training data

{
  "train_groups": 1184,
  "gold_train_groups": 48,
  "test_groups": 45,
  "gold_test_groups": 16,
  "train_candidates": 27544,
  "train_pass": 5567,
  "train_fail": 21977,
  "pass_ratio": 0.202,
  "avg_candidates_per_train_group": 23.3,
  "avg_pass_per_train_group": 4.7
}

Trained 2026-07-07T07:27:10 · source: https://github.com/s3777091/recursive_models