bio2nl retained model bank for the October revision
This newly created repository retains all prescribed seeds of the main October transfer routes and their associated diagnostics. It is separate from the older dnagpt/bio2nl-models repository. Data, source definitions, training code and per-seed result tables are in dnagpt/bio2nl-data-2026-10.
Upload state: use publication_status.json to distinguish the planned catalog from weights already committed. Until that file says complete_verified, some catalog paths may still be uploading.
| Model family | Retained checkpoints | Source / seeds |
|---|---|---|
| Forward official GPT-2 | 100 | Clean PAWS-X, literal SFT seeds 1–100 |
| Ordinary reverse GPT-2 | 100 | Four BLAST identity groups + mixed, 20 seeds each |
| Joint from-scratch pretraining | 3 | EP/ES/EE, pretraining seed 0 once each |
| Joint source SFT | 60 | Same high-identity V2 protein group, 20 SFT seeds per pretrained condition |
| Remote source | 20 | SCOPe, seed 1 discovery + seeds 2–20 replication |
| Functional-variant source | 20 | Four epochs, seed 1 discovery + seeds 2–20 replication |
| Retained diagnostics | 6 | Alternative LR and DMS full/small 20-epoch fits, one seed |
| Fixed-recipe repeats | 4 | Two ordinary seed-19 recipes, two repeats each |
Total: 313 complete checkpoint files, about 141.72 GiB. Initial classification heads are retained separately where originally saved. Reused classifiers and repeated inference are not new training repetitions. Fine-tuning seeds are not independent pretraining seeds.
Loading one model
Download model_catalog.json, code/, the chosen models/<model_id>/ folder and the tokenizer referenced in its record. Native256 models use the native GPT-2 pair API without an inserted separator and built-in pooling. Remote/DMS native1024 models require explicit attention-mask final-token pooling; using default EOS-as-pad pooling would change the classifier. Joint models use a new 32k BPE and final attended EOS with 512-token input. code/load_models.py implements these three formats; code/joint_models.py is the exact original complete-state loader.
import sys
sys.path.insert(0, '/path/to/downloaded/bank/code')
from load_models import load_model, encode_pairs, logits
model, record = load_model('/path/to/downloaded/bank', 'ordinary/id80_100/seed_01')
batch = encode_pairs('/path/to/downloaded/bank', record, [('ACDEFGHIK', 'ACDEYGHIK')])
scores = logits(model, record, batch)
This example checks loading, not benchmark performance. Reproduce the frozen numerical environment for exact scores (recorded per model; principal later runs use PyTorch2.5.1+cu121/Transformers4.51.3). Native weights use safetensors. Joint safe.pt files contain tensors plus primitive JSON metadata and are loaded with weights_only=True; they are not Hugging Face AutoModel checkpoint folders. The original weight bytes and complete classifier heads are preserved.
The official backbone is openai-community/gpt2 revision 607a30d783dfa663caf39e06633721c8d4cfcd7e. Joint models are independently trained project models with one pretraining seed per condition. English validation labels may fix class orientation, which must then stay fixed on test/confirmation. No English gradient updates are used in protein-source SFT. Model performance is exploratory and task-dependent; source training fit is distinct from source generalization. See LICENSES.md for attribution.