Instructions to use openai/gpt-oss-20b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openai/gpt-oss-20b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openai/gpt-oss-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")
model = AutoModelForCausalLM.from_pretrained("openai/gpt-oss-20b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use openai/gpt-oss-20b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openai/gpt-oss-20b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openai/gpt-oss-20b

SGLang

How to use openai/gpt-oss-20b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openai/gpt-oss-20b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openai/gpt-oss-20b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openai/gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openai/gpt-oss-20b with Docker Model Runner:
```
docker model run hf.co/openai/gpt-oss-20b
```

No matching torch==2.9.0.dev20250804+cu128 wheel for Python 3.12 – cannot install vllm==0.10.1+gptoss with official instructions

#165

by Sneha7 - opened Nov 6, 2025

Discussion

Sneha7

Nov 6, 2025

•

edited Nov 6, 2025

Hi team,

When following the official installation instructions for vllm==0.10.1+gptoss with Python 3.12 and CUDA 12.2, the installation fails because there is no torch==2.9.0.dev20250804+cu128 wheel published for cp312 (Python 3.12) in the referenced extra index (https://download.pytorch.org/whl/nightly/cu128/).

Steps to reproduce:

uv pip install --system --pre vllm==0.10.1+gptoss  --extra-index-url https://wheels.vllm.ai/gpt-oss/  --extra-index-url https://download.pytorch.org/whl/nightly/cu128  --index-strategy unsafe-best-match

Error:

ERROR: No matching distribution found for torch==2.9.0.dev20250804+cu128

Python version: 3.12
CUDA: 12.2
OS: (e.g., Ubuntu 22.04, cloud/container)
vllm: 0.10.1+gptoss

Expected behavior

Install should succeed following official instructions, as Python 3.12 is the officially supported version for gpt-oss.

Request

Please publish torch==2.9.0.dev20250804+cu128 wheels for Python 3.12 in the nightly or gpt-oss index, or clarify which torch version should be used for Python 3.12.
If not supported, please update README/installation instructions or provide the expected timeline for Python 3.12 support.

Thanks!

kavyamali

Nov 8, 2025

torch==2.9.0.dev20250804+cu128 wheels do not exist for Python 3.12 at the moment, but they do for Python 3.11. You can downgrade to Python 3.11 and retry installing torch in venv.

kopemon01

Nov 23, 2025

Tried with Python 3.11 but no luck. torch==2.9.0.dev20250804+cu128 not found. I think the package index is changed and it is gone.

$ uv pip install --pre vllm==0.10.1+gptoss --extra-index-url https://wheels.vllm.ai/gpt-oss/ --extra-index-url https://download.py
torch.org/whl/nightly/cu128 --index-strategy unsafe-best-match
Using Python 3.11.14 environment at: .venv3.11
⠴ numba==0.61.2 × No solution found when resolving dependencies:
╰─▶ Because there is no version of torch==2.9.0.dev20250804+cu128 and vllm==0.10.1+gptoss depends on
torch==2.9.0.dev20250804+cu128, we can conclude that vllm==0.10.1+gptoss cannot be used.
And because you require vllm==0.10.1+gptoss, we can conclude that your requirements are unsatisfiable.

kavyamali

Nov 24, 2025

•

edited Nov 24, 2025

Tried with Python 3.11 but no luck. torch==2.9.0.dev20250804+cu128 not found. I think the package index is changed and it is gone.

$ uv pip install --pre vllm==0.10.1+gptoss --extra-index-url https://wheels.vllm.ai/gpt-oss/ --extra-index-url https://download.py
torch.org/whl/nightly/cu128 --index-strategy unsafe-best-match
Using Python 3.11.14 environment at: .venv3.11
⠴ numba==0.61.2 × No solution found when resolving dependencies:
╰─▶ Because there is no version of torch==2.9.0.dev20250804+cu128 and vllm==0.10.1+gptoss depends on
torch==2.9.0.dev20250804+cu128, we can conclude that vllm==0.10.1+gptoss cannot be used.
And because you require vllm==0.10.1+gptoss, we can conclude that your requirements are unsatisfiable.

Tried searching the wheels manually, and it seems the new wheels might be at https://download.pytorch.org/whl/nightly/, divided into torch, torchaudio and torchvision, because the cu128 folder redirects
back to the same link. The old ones were most likely removed, or their indexing was changed.
Also found these:

Torchvision:
https://download.pytorch.org/whl/nightly/cu128/torchvision-0.25.0.dev20250926%2Bcu128-cp312-cp312-win_amd64.whl
https://download.pytorch.org/whl/nightly/cu128/torchvision-0.25.0.dev20250926%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl

Torch:
https://download.pytorch.org/whl/nightly/cu128/torch-2.10.0.dev20250925%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl
https://download.pytorch.org/whl/nightly/cu128/torch-2.10.0.dev20250925%2Bcu128-cp312-cp312-win_amd64.whl

Torchaudio:
https://download.pytorch.org/whl/nightly/cu128/torchaudio-2.10.0.dev20251018%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl
https://download.pytorch.org/whl/nightly/cu128/torchaudio-2.10.0.dev20251018%2Bcu128-cp312-cp312-win_amd64.whl

Compatible python 3.11 (cp311) wheels were also present, you may try installing them in the venv.

this might only work with --no-deps attribute, although if it errors for the dependency not proper to be used, I think vllm won't work till the new indexes are known (if they exist).
Note that this is still a hopeful solution, just a workaround with 2.10.0 torch, I haven't tested it. It might just fail at runtime.

I would, though, advice using a docker build, instead of what's stated above.

UPDATE: Found these wheels.
https://download.pytorch.org/whl/cu128/torch-2.9.0%2Bcu128-cp312-cp312-win_amd64.whl#sha256=c97dc47a1f64745d439dd9471a96d216b728d528011029b4f9ae780e985529e0
https://download.pytorch.org/whl/cu128/torch-2.9.0%2Bcu128-cp312-cp312-manylinux_2_28_x86_64.whl#sha256=87c62d3b95f1a2270bd116dbd47dc515c0b2035076fbb4a03b4365ea289e89c4

These have a better chance of working.

iijangjang

Dec 17, 2025

Currently, there is no longer a need to install using the branch vllm==0.10.1+gptoss. The latest version of vllm can be used. I can run it successfully by installing with the following command:
pip install --pre vllm --extra-index-url https://wheels.vllm.ai/gpt-oss/

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment