Instructions to use sofisticated/broken-model-update with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sofisticated/broken-model-update with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sofisticated/broken-model-update")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sofisticated/broken-model-update")
model = AutoModelForCausalLM.from_pretrained("sofisticated/broken-model-update", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sofisticated/broken-model-update with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sofisticated/broken-model-update"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sofisticated/broken-model-update",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sofisticated/broken-model-update

SGLang

How to use sofisticated/broken-model-update with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sofisticated/broken-model-update" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sofisticated/broken-model-update",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sofisticated/broken-model-update" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sofisticated/broken-model-update",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sofisticated/broken-model-update with Docker Model Runner:
```
docker model run hf.co/sofisticated/broken-model-update
```

broken-model-update / README.md

sofisticated

added HF url

42e27a8 6 months ago

preview code

Raw

History Blame Contribute Delete

2.69 kB

metadata

library_name: transformers
pipeline_tag: text-generation
base_model:
  - Qwen/Qwen3-8B-Base

https://huggingface.co/sofisticated/broken-model-update

A

The original configuration files indicated both Qwen and Llama models. Looking at the safetensors files the actual model is Qwen3. I then compared this model's config files to those of Qwen/Qwen3-8B and made the following changes:

added a chat_template field to the tokenizer_config file and copied over the value used in the Qwen/Qwen3-8B model
changed base_model in the README.md yaml from meta-llama/Meta-Llama-3.1-8B to Qwen/Qwen3-8B-Base

Per the blog article: (6 Common Pitfalls in Sharing Models on Hugging Face)[https://friendli.ai/blog/common-pitfalls-in-sharing-models-on-hugging-face]: "Hugging Face’s chat interface relies on a chat template to format inputs. Without it, your chat model may not work as expected."

The other files and fields are consistent with the Qwen/Qwen3-8B model.

B

reasoning_effort has no effect because neither the model nor the serving stack maps that parameter to any behavior. To make it meaningful, the engine must interpret reasoning_effort and adjust routing, prompts, or decoding, and ideally the model must be trained or fine-tuned to take advantage of these different ‘effort levels’.

To make reasoning_effort actually matter, you need changes at both serving and model/training levels:

Serving-layer changes

Parse the reasoning_effort field in the /chat/completions request.
Map the value to concrete inference-time behaviors, for example:
- Adjust max_tokens, temperature, and top_p to allow longer or more exploratory chains of thought when reasoning_effort is high.
- Switch to a different prompt template that includes explicit chain-of-thought instructions or additional scratchpad tokens for higher effort.
- Route to a different model variant (e.g., “reasoning” vs “normal”) if you host multiple checkpoints.
Ensure the OpenAI-compatible server you’re using (Friendli Engine or vLLM-based gateway) actually forwards/consumes this parameter instead of dropping it.

Model/training changes

Train or fine-tune the model so it can exploit the extra context or tokens provided for “higher effort”:
- Instruction tuning with CoT data and different levels of reasoning depth.
- Possibly multi‑expert or MoE heads where “high effort” can select a more compute-intensive branch.
For truly semantic reasoning levels (like Anthropic’s “reasoning” modes), you might need specialized training regimes that align different effort levels with different internal behaviors, not just longer responses.