mhmsadegh/rectom-causal-reasoning-crossencoder-bce-v2-709items

RecToM Causal-Reasoning Cross-Encoder v2 (BCE, 709-item training pool)

Same BCE cross-encoder recipe as rectom-causal-reasoning-crossencoder-bce (the {'MATCH@1': 0.064, 'Hit@1': 0.064, 'Hit@3': 0.218, 'Hit@5': 0.305, 'MRR': 0.179} 266-item model), retrained on a 709-item leakage-safe Case Bank (docs129 large-scale expansion) to test whether scaling causal-path supervision improves reasoning-aware retrieval over frozen semantic (BGE) retrieval.

Group-disjoint 5-fold CV results (bare-dialogue_id grouping, fixing a grouping-key bug found in docs129): { "crossencoder_frozen": { "n": 709, "MATCH@1": 0.06770098730606489, "Hit@1": 0.06770098730606489, "Hit@3": 0.17489421720733428, "Hit@5": 0.2679830747531735, "MRR": 0.16755665214761337, "per_task_hit3": { "belief": 0.49019607843137253, "desire": 0.18110236220472442, "judgement": 0.10483870967741936, "prediction": 0.09375, "intention": 0.11764705882352941 }, "per_task_n": { "belief": 102, "desire": 127, "judgement": 248, "prediction": 96, "intention": 136 } }, "crossencoder_finetuned": { "n": 709, "MATCH@1": 0.08180535966149506, "Hit@1": 0.08180535966149506, "Hit@3": 0.2059238363892807, "Hit@5": 0.2947813822284908, "MRR": 0.1802352172741932, "per_task_hit3": { "belief": 0.3333333333333333, "desire": 0.14960629921259844, "judgement": 0.22580645161290322, "prediction": 0.07291666666666667, "intention": 0.22058823529411764 }, "per_task_n": { "belief": 102, "desire": 127, "judgement": 248, "prediction": 96, "intention": 136 } } }

McNemar (fine-tuned vs frozen): {'ft_only_wins': 100, 'frozen_only_wins': 78, 'n_discordant': 178, 'p_value': np.float64(0.11523195131755465)} Bootstrap 95% CI (Hit@3 delta): {'point_delta': 0.03102961918194641, 'ci_95_lo': -0.004231311706629048, 'ci_95_hi': 0.06770098730606489}

No causal path, gold answer, Direct/Causal outcome, or Benefit/Harm label is ever used as model input -- supervision-only, via the pair label.