Yotto3108/cgm-joint-routing-3500

CGM joint-routing baseline: 3,500 updates

Research handoff for Udit. This repository contains the original single-path, 3,500-update baseline, seed 43 only. This is the best baseline seed by the existing PR-AUC evaluation and provides the established 0.584241857 PR-AUC reference. The later recovery/exposure models, ensembles and downstream knowledge-distillation classifiers are not included.

What is included

• Original checkpoints/seed_43/model.pt, unchanged and checksum-verified.
• All non-Chronos model state: the CGM encoder, fusion, routing, input embeddings, prediction/reconstruction heads and EMA state, plus the original AdamW state.
• The exact model implementation and required local imports under source/.
• load_cgm.py, bundle.json, dependency versions and numerical verification.

The encoder is not just jepa.online_encoder. Its evaluated 128D representation also depends on the trained fusion/router/input embedding and frozen Chronos-2. The loader downloads the exact frozen amazon/chronos-2 revision recorded in bundle.json and checks both backbone files. These approximately 478 MB of upstream weights are not duplicated here. Native Chronos weights are needed for inference as well as training. Upstream terms remain with the upstream model repository.

No CGM records, patient-level predictions, labels or cached patient embeddings are distributed. Aggregate metrics are developmental results, not clinical validation.

Download and setup

Use an isolated Python environment and install the appropriate PyTorch build for your platform first. The export was verified with PyTorch 2.11.0+cu126; exact dependency versions are in requirements.txt. Different hardware/software may change floating-point results.

hf download Yotto3108/cgm-joint-routing-3500 --local-dir cgm-joint-routing-3500
cd cgm-joint-routing-3500
python -m pip install -r requirements.txt
python load_cgm.py --seed 43 --device cuda

For a reproducible experiment, record the Hub commit SHA and add --revision <commit-sha> to the download command. Python snapshot_download is also supported. This is a PyTorch research bundle, not a Transformers AutoModel.

Extract the ordinary 128D representation

import torch
from load_cgm import load_cgm

model, info, _ = load_cgm(seed=43, device="cuda")
# One 24-hour window: 288 five-minute slots. Values are raw mg/dL.
x = torch.full((2, 288), 110., device="cuda")  # synthetic example only
observed = torch.ones_like(x, dtype=torch.bool)
# Five-minute position of the first sample in its local day, in [0, 287].
start_index = torch.zeros(2, dtype=torch.long, device="cuda")
features = model.features(x, observed, start_index)  # [2, 128], no gradients

Retain the observation mask when handling missing values. Each window must contain observed finite values. Do not pre-normalize mg/dL inputs or fabricate observations at unmeasured points. Apply the existing dataset preprocessing and timestamp conventions for comparable evaluation. A constant synthetic input checks loading; it is not a performance test.

Start a new pretraining or fine-tuning experiment

load_cgm keeps the original trainable/frozen parameter flags. It returns eval mode; call model.train() before training. Use model.representation(...) for a differentiable 128D output; model.features(...) intentionally disables gradients.

model, info, _ = load_cgm(seed=43, device="cuda")
optimizer = model.optimizer()  # fresh optimizer for a NEW experiment
model.train()
step = info["step"]
values, visible, hidden = model.prepare(x, observed, step)
optimizer.zero_grad(set_to_none=True)
loss, stats = model.loss_prepared(values, visible, start_index, hidden, step)
loss.backward()
router = list(model.router.parameters())
router_ids = {id(p) for p in router}
other = [p for p in model.parameters() if p.requires_grad and id(p) not in router_ids]
torch.nn.utils.clip_grad_norm_(other, 1., error_if_nonfinite=True)
torch.nn.utils.clip_grad_norm_(router, 1., error_if_nonfinite=True)
optimizer.step()
model.update_targets()

This is a single-batch wiring example for the original objective, not a prescribed new research budget. Define the data split, update budget and learning rates before a full experiment. The backbone, EMA targets and reconstruction verifiers remain frozen as in the original recipe.

For the saved AdamW moments instead of a fresh optimizer:

model, info, optimizer = load_cgm(seed=43, device="cuda", restore_optimizer=True)

Optimizer and model state are present, but the original checkpoint does not store all RNG/sampler positions. Do not call this bitwise-identical uninterrupted training. Record whether optimizer state was restored or reset.

Baseline and evaluation boundaries

The encoder was trained on 1,094 public pretraining windows without WearCGM. The reported seed-43 result is ordinary frozen-encoder linear probing: PR-AUC 0.584241857, AUROC 0.664719478, macro-F1 0.599706888. Evaluation uses 2,699 windows, 14 dataset/task cells, five subject folds and ten repeats, fold seed 42; the table average is over dataset/task cells. It is not a three-seed mean or an ensemble result. See bundle.json for provenance and verified metrics.

This checkpoint was selected during repeated development evaluation. Existing cross-session participant overlap in the source protocol remains; this is not an untouched, universally participant-disjoint final-test claim. Keep the same ordinary LP protocol when measuring encoder improvements. Report ensembles or downstream classifier KD as separate conditions.

No downstream classifier is included: Udit should train the probe or downstream head within the training portion of each split. Keep outcome labels out of pretraining teacher selection and representation learning.