CrossGAT Stage-B — LambdaRank + retrieval negatives

Two-class eligibility model (excluded vs eligible) trained with a LambdaRank objective and retrieval-distributed negatives.

Output classes: excluded / eligible. These are the five cross-validation checkpoints — one per fold, each trained on 80% of the topics and never shown its own fold's.

⚠️ Not self-contained

This model does not take raw text. It scores parsed eligibility graphs — entity/relation structures extracted from trial criteria and patient descriptions by an LLM. The graphs for all 13,229 TREC CT trials are published separately: clinical-trials-eligibility-graphs-rerank.

What it is for

It reranks, it does not retrieve. Trained on an eligibility target, it reorders within the candidate set a first-stage retriever already found. It cannot judge whether a trial is even on the right disease — per-topic AUC for relevant-vs-rest is 0.533, i.e. chance. Fuse it with the retrieval score, never replace it: the pure-model ranking (w=1.0 below) is consistently worse than the first stage.

Benchmark

Fold-consistent 5-fold CV over 162 TREC CT 2021-23 topics, hybrid BM25 + embeddinggemma-300m-medical first stage, top-100 candidates, scored by p_eligible and fused convexly with the retrieval score. p from a paired Wilcoxon signed-rank test against the first stage (zero_method='zsplit', so topics left unchanged count as evidence of no effect).

configuration	graded NDCG@10	Δ vs first stage	p	judged@10
first stage only	0.5973	—	—	0.922
fusion w=0.3	0.6195	+0.0221	0.00335	0.911
fusion w=0.5	0.6257	+0.0284	0.0068	0.878
fusion w=0.7	0.5969	-0.0004	0.841	0.809
fusion w=1	0.5394	-0.0580	0.0138	0.730

Graded gain is 2^label - 1 (0/1/3). Binary NDCG is not the metric to read here: excluded and eligible both count as relevant under it, so the ordering this model is trained to produce is invisible to it by construction.

Usage

pip install "crossgat @ git+https://github.com/JDev2001/msc_v2"

from crossgat_hf import load, score, rerank_score
pipe = load("2001jdev/crossgat-trial-eligibility-2class-rank", fold=0, metric="ndcg")
out  = score(pipe, patient_graph, inc_graph, exc_graph,
             patient_text="65yo male, type 2 diabetes, HbA1c 8.1%",
             trial_text="Metformin in adults with type 2 diabetes ...")
rerank_score(out, "p_eligible")

This is a graph-only checkpoint with no text branch; patient_text / trial_text are accepted and ignored.

metric="ndcg" selects on val_ndcg10 and is what every reported number uses; metric="f1" selects on val_macro_f1, classifies better, and has not been benchmarked for reranking.

Checkpoint provenance

checkpoint_manifest.json maps each released file to its original training filename with a sha256 prefix. Checkpoints were resolved by modification time, not by the epoch in the filename — several training runs can share a fold directory.

crossgat-trial-eligibility-cv — 3-class LambdaRank + hard negatives, the best reranker of this family
crossgat-trial-eligibility-full — post-CV refit for deployment

Downloads last month: -; Downloads are not tracked for this model. How to track