Instructions to use claytonwang/rlcr-qwen3-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use claytonwang/rlcr-qwen3-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="claytonwang/rlcr-qwen3-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("claytonwang/rlcr-qwen3-8b")
model = AutoModelForCausalLM.from_pretrained("claytonwang/rlcr-qwen3-8b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use claytonwang/rlcr-qwen3-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "claytonwang/rlcr-qwen3-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claytonwang/rlcr-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/claytonwang/rlcr-qwen3-8b

SGLang

How to use claytonwang/rlcr-qwen3-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "claytonwang/rlcr-qwen3-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claytonwang/rlcr-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "claytonwang/rlcr-qwen3-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claytonwang/rlcr-qwen3-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use claytonwang/rlcr-qwen3-8b with Docker Model Runner:
```
docker model run hf.co/claytonwang/rlcr-qwen3-8b
```

Qwen3 Causal Language Model (BF16)

This repository contains a decoder-only causal language model using the Qwen3 architecture. The weights are stored in sharded Safetensors format using BF16 precision.

Model details

Item	Value
Architecture	`Qwen3ForCausalLM`
Model type	`qwen3`
Parameters	8,190,735,360
Weight precision	BF16
Hidden size	4096
Layers	36
Attention heads	32
Key/value heads	8
Vocabulary size	151,936
Configured context length	40,960 tokens
Weight format	7 sharded Safetensors files (about 5 GB each)

The tokenizer configuration currently declares a maximum length of 131,072, while the model configuration declares 40,960 positions. Until the longer context length is independently validated, use 40,960 tokens as the supported maximum.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
)

messages = [{"role": "user", "content": "你好，请介绍一下你自己。"}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=256)

answer = output[0, inputs.input_ids.shape[1]:]
print(tokenizer.decode(answer, skip_special_tokens=True))

BF16 weights require roughly 16 GB just for the model parameters, plus additional memory for inference state and the KV cache. Use a machine with sufficient RAM or accelerator memory. The example selects SDPA so that FlashAttention 2 is not required; installations with FlashAttention 2 may choose attn_implementation="flash_attention_2" instead.

Training and provenance

The files supplied with this model do not identify the following information:

Base model or checkpoint
Training or fine-tuning method
Training datasets
Evaluation results
Model author or organization

The uploader should add these details before publishing so users can assess the model's capabilities, limitations, and provenance.

License

No license information was included with the supplied model files. The uploader must verify the licenses of the base model, training data, and resulting weights and add the correct license before making the repository public. Absence of a license does not grant permission to use, modify, or redistribute the model.

Limitations

No benchmark, safety, bias, or long-context evaluation results were supplied. Do not rely on this model for high-impact decisions without appropriate testing and human review.

Downloads last month: 285

Safetensors

Model size

8B params

Tensor type

BF16