Bio-posttrain Qwen3-4B CPT

Continued pre-training (CPT) checkpoint from How Post-Training Shapes Biological Reasoning Models.

Model details

  • Base model: Qwen/Qwen3-4B
  • Stage: CPT (text-only omics corpus)
  • Training: lr=1e-5, gradient accumulation=64, final CPT checkpoint

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("mims-harvard/bio-posttrain-qwen3-4b-cpt", trust_remote_code=True)

Collection

Part of the Bio-posttrain collection on Hugging Face.

Citation

@article{bio_posttrain_2026,
  title={How Post-Training Shapes Biological Reasoning Models},
  author={...},
  year={2026}
}
Downloads last month
12
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for mims-harvard/bio-posttrain-qwen3-4b-cpt

Finetuned
Qwen/Qwen3-4B
Finetuned
(735)
this model

Collection including mims-harvard/bio-posttrain-qwen3-4b-cpt