Instructions to use PawanKrd/RastNus-Base-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PawanKrd/RastNus-Base-12B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PawanKrd/RastNus-Base-12B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("PawanKrd/RastNus-Base-12B")
model = AutoModelForMultimodalLM.from_pretrained("PawanKrd/RastNus-Base-12B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PawanKrd/RastNus-Base-12B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PawanKrd/RastNus-Base-12B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Base-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PawanKrd/RastNus-Base-12B

SGLang

How to use PawanKrd/RastNus-Base-12B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PawanKrd/RastNus-Base-12B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Base-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PawanKrd/RastNus-Base-12B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PawanKrd/RastNus-Base-12B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PawanKrd/RastNus-Base-12B with Docker Model Runner:
```
docker model run hf.co/PawanKrd/RastNus-Base-12B
```

Issues running RastNus-Base-12B for Text-Only Kurdish Grammar

by aspava - opened 29 days ago

Discussion

aspava

29 days ago

Error Correction

Hi Pawan,

I am testing the PawanKrd/RastNus-Base-12B model locally using HuggingFace Transformers for Kurdish Grammar Error Correction (GEC).

I am encountering non-deterministic and erratic outputs when passing text-only prompts. Here are the specific issues:

Coordinate / Tag Output: When prompted with simple Kurdish text, the model often returns bounding box coordinates or verification tags (e.g., 4 (5), Rast, 1 (0), R-C1234...) instead of corrected natural language sentences.

Base / Chat Template Behavior: Using processor.apply_chat_template or custom GEC prompts (Şaş: ... \nRast:) results in empty HTML tags (...) or classification labels rather than sequence-to-sequence text rewriting.

Pipeline: I am running the model with bitsandbytes 4-bit quantization on an RTX 4050 (6GB VRAM) via AutoModelForMultimodalLM and AutoProcessor.

Could you please clarify:

What is the exact expected prompt/template structure or task prefix for text-only GEC?

Is an input image mandatory due to the underlying PaliGemma-2 architecture?

Was this model fine-tuned as an end-to-end text rewriter, or is it strictly a error detection / token-tagging classifier?

Thanks for your work on Kurdish NLP!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment