Instructions to use kothasuhas/infinite_compute_distillation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kothasuhas/infinite_compute_distillation with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kothasuhas/infinite_compute_distillation")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kothasuhas/infinite_compute_distillation")
model = AutoModelForCausalLM.from_pretrained("kothasuhas/infinite_compute_distillation", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kothasuhas/infinite_compute_distillation with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kothasuhas/infinite_compute_distillation"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kothasuhas/infinite_compute_distillation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/kothasuhas/infinite_compute_distillation

SGLang

How to use kothasuhas/infinite_compute_distillation with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kothasuhas/infinite_compute_distillation" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kothasuhas/infinite_compute_distillation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kothasuhas/infinite_compute_distillation" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kothasuhas/infinite_compute_distillation",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use kothasuhas/infinite_compute_distillation with Docker Model Runner:
```
docker model run hf.co/kothasuhas/infinite_compute_distillation
```

Infinite compute distillation checkpoints

The original context-16 d8 teacher is at the repository root. All 170 fully evaluated student checkpoints available at snapshot 2026-10-27T22:02:26.133341+00:00 are under students/<method>/<run>/. Every folder includes exact original FP32 weights/config, the matching 4096-token tokenizer, and portable measured metrics for students. model_index.json lists paths, model/data provenance, file SHA256 values, horizon, LR, seed, tune NLL, and held-out test KL.

The frozen random-target control is also published: the original seed-42 d8 standard initialization, with zero training updates. Its matched tokenizer and manifest accompany the exact weights.

Method	Checkpoints
adamh	32
adamw_bs1024	25
adamw_logits_from_teacher	13
adamw_logits_seed42	25
adamw_seed123	25
adamw_seed42	25
muonh	25
random_target	1
teacher	1

The teacher has 8 Llama layers, width 512, MLP width 1792, 8 heads, head dimension 64, untied embeddings, and no QK norm (34,611,712 parameters). It was trained from scratch on 614.4M DCLM payload tokens. It is a context-16 research model, not a chat/instruction model. Student architectures match it exactly. Training/inference used FP32 weights and BF16 autocast. The original tokenizer source is pinned at 23dbd5ab0fcc1edfd25eedf1ae9d41a5d8b6b233; its SentencePiece bytes match the original cached tokenizer used for DCLM preparation.

Teacher weights SHA256: 1337eaec061a873932eaa6de968aa113f221334dd6c3de56d6f09006f06387fe.

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "kothasuhas/infinite_compute_distillation"
teacher = AutoModelForCausalLM.from_pretrained(repo)
tokenizer = AutoTokenizer.from_pretrained(repo)
# For a student, pass subfolder="students/<method>/<run>" to both loaders.

In the experiment code repository:

uv run --frozen --extra data python -m scripts.fetch_teacher
uv run --frozen --extra data python -m scripts.fetch_models --list
uv run --frozen --extra data python -m scripts.fetch_models --best-method adamw_logits_seed42 --tokens 1006632960

Downloads are public and hash-verified from an immutable revision. Sample generation in the research code uses the native PyTorch loop, with no HF or vLLM generation. The original teacher samples and prepared DCLM arrays are also public.

The index's best-measured entries select real trained checkpoints using full tune sample NLL at each horizon. They are distinct from the plotted intermediate LR estimates, for which no checkpoint was trained. Methods still running at this snapshot: adamw_logits_from_teacher. Pending/incomplete runs and diverged runs are excluded; rerun the publisher to add newly evaluated checkpoints.

Downloads last month: -

Safetensors

Model size

34.6M params

Tensor type

F32

kothasuhas
/

infinite_compute_distillation

Infinite compute distillation checkpoints

Dataset used to train kothasuhas/infinite_compute_distillation