Instructions to use ftajwar/d24-climbmix-dolmino-midtrain-100b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ftajwar/d24-climbmix-dolmino-midtrain-100b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ftajwar/d24-climbmix-dolmino-midtrain-100b")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ftajwar/d24-climbmix-dolmino-midtrain-100b")
model = AutoModelForCausalLM.from_pretrained("ftajwar/d24-climbmix-dolmino-midtrain-100b")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ftajwar/d24-climbmix-dolmino-midtrain-100b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ftajwar/d24-climbmix-dolmino-midtrain-100b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ftajwar/d24-climbmix-dolmino-midtrain-100b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ftajwar/d24-climbmix-dolmino-midtrain-100b

SGLang

How to use ftajwar/d24-climbmix-dolmino-midtrain-100b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ftajwar/d24-climbmix-dolmino-midtrain-100b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ftajwar/d24-climbmix-dolmino-midtrain-100b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ftajwar/d24-climbmix-dolmino-midtrain-100b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ftajwar/d24-climbmix-dolmino-midtrain-100b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ftajwar/d24-climbmix-dolmino-midtrain-100b with Docker Model Runner:
```
docker model run hf.co/ftajwar/d24-climbmix-dolmino-midtrain-100b
```

d24-climbmix-dolmino-midtrain-100b

A ~0.75B-parameter (756,819,456) decoder-only LM. This is the d24-climbmix-100b base continue-pretrained ("midtrained") for another ~100B tokens on a high-quality reasoning/instruction blend with pretrain-data replay.

Base: d24 pretrained from scratch on 100B tokens of ClimbMix.
Midtrain: +100B tokens as a second pretraining phase — weights-only reload of the base, fresh optimizer, re-warmed to the original peak LR and re-decayed (Ibrahim et al. 2024; Gupta et al. 2023).
Data mixture (true per-batch blend): 80% OLMo-3 Dolmino (reasoning/instruction, at its final 24-component proportions) + 20% ClimbMix replay (the base's exact pretrain distribution, to anchor against catastrophic forgetting).

This is a base model (no instruction/chat tuning, no chat template) intended for research and further SFT/RL. Total training ≈ 200B tokens.

Architecture


Layers	24
Hidden	1536
Heads	12 (MHA)
FFN	4096 (SwiGLU / silu)
Pos. enc.	RoPE (θ=10000)
Norm	RMSNorm
Embeddings	tied
Vocab	50304 (gpt2 tokenizer)
Seq len	2048
Dtype	bf16

Midtrain hyperparameters

Global batch 1024 · seq 2048 · 47,684 steps (~100B tokens) · AdamW β(0.9, 0.95), wd 0.1, grad-clip 1.0 · peak LR 3e-4 → min 3e-5, cosine · warmup 2000 steps. Peak LR validated stable via an LR-ablation smoke.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
mid = "ftajwar/d24-climbmix-dolmino-midtrain-100b"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype=torch.bfloat16).cuda().eval()
ids = tok("The first three prime numbers are", return_tensors="pt").to("cuda")
print(tok.decode(model.generate(**ids, max_new_tokens=40)[0], skip_special_tokens=True))

Tokenizer

gpt2 BPE (vocab padded to 50304), --append-eod during data prep. It is a base completion model — prompt it with plain text, not chat turns.

Downloads last month: 450

Safetensors

Model size

0.8B params

Tensor type

BF16

Model tree for ftajwar/d24-climbmix-dolmino-midtrain-100b

Finetunes

2 models