Instructions to use SHSLab/Step-5-Preview-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SHSLab/Step-5-Preview-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="SHSLab/Step-5-Preview-BF16", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("SHSLab/Step-5-Preview-BF16", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SHSLab/Step-5-Preview-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SHSLab/Step-5-Preview-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SHSLab/Step-5-Preview-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/SHSLab/Step-5-Preview-BF16

SGLang

How to use SHSLab/Step-5-Preview-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SHSLab/Step-5-Preview-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SHSLab/Step-5-Preview-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SHSLab/Step-5-Preview-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SHSLab/Step-5-Preview-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use SHSLab/Step-5-Preview-BF16 with Docker Model Runner:
```
docker model run hf.co/SHSLab/Step-5-Preview-BF16
```

Request: DOI

by win10 - opened about 20 hours ago

Discussion

win10

about 20 hours ago

Fake storage

khtsly

about 17 hours ago

its not

win10

about 16 hours ago

its not

The official statement indicates that the content will be open-sourced on October 15th, and this lab is not official at all!

khtsly

about 14 hours ago

•

edited about 14 hours ago

i know its not official, but they actually accidentally set it as public for ~1 hour. so many people get attention, and some of them immediately forked or downloaded it (the files) before it get privated back.

khtsly

about 14 hours ago

•

edited about 14 hours ago

and the weights & stuff here or in huggingface models, are real. can you use/load it? probably not. why? they didnt expose the modeling py file. stepfun modifies transformers package by themself to support this new stepfun 5 preview model architectures, and they also implement their own kernel optimizations (e.g. triton) which arent publicly available.

khtsly

about 14 hours ago

but as an reverse engineer with AI/ML knowledged, they could just do it.

win10

about 14 hours ago

i know its not official, but they actually accidentally set it as public for 1 hour. so many people get attention and some of them forked & downloaded it before it get privated back.

I know that, but I hope this leaked model will persist until it's saved on my disk. Such a targeted naming will attract the attention of the authorities.