Instructions to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pankajpandey-dev/qwen3.5-9b-hindi-instruct")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("pankajpandey-dev/qwen3.5-9b-hindi-instruct")
model = AutoModelForMultimodalLM.from_pretrained("pankajpandey-dev/qwen3.5-9b-hindi-instruct")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pankajpandey-dev/qwen3.5-9b-hindi-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pankajpandey-dev/qwen3.5-9b-hindi-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/pankajpandey-dev/qwen3.5-9b-hindi-instruct

SGLang

How to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pankajpandey-dev/qwen3.5-9b-hindi-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pankajpandey-dev/qwen3.5-9b-hindi-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pankajpandey-dev/qwen3.5-9b-hindi-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pankajpandey-dev/qwen3.5-9b-hindi-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pankajpandey-dev/qwen3.5-9b-hindi-instruct to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for pankajpandey-dev/qwen3.5-9b-hindi-instruct to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for pankajpandey-dev/qwen3.5-9b-hindi-instruct to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="pankajpandey-dev/qwen3.5-9b-hindi-instruct",
    max_seq_length=2048,
)

Docker Model Runner
How to use pankajpandey-dev/qwen3.5-9b-hindi-instruct with Docker Model Runner:
```
docker model run hf.co/pankajpandey-dev/qwen3.5-9b-hindi-instruct
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.5-9B Hindi Instruct 🇮🇳

Text-only fine-tune of a multimodal base — use for text generation as shown below (ignore the auto-suggested image examples).

Qwen3.5-9B fine-tuned to answer directly in fluent, native Hindi — no English code-switching, no English "thinking" detours. Runs on a laptop CPU via the GGUF version (~5.7 GB).

Why this exists

Base Qwen3.5-9B knows Hindi, but ask it a Hindi question and it burns hundreds of tokens thinking in English inside its think block before answering — and its answers code-switch. This fine-tune closes the think block immediately and replies in pure Devanagari, following instructions tightly.

Examples (from the release eval)

प्रश्न: अपनी माँ के लिए धन्यवाद व्यक्त करते हुए चार पंक्तियाँ लिखिए।

उत्तर: माँ, शब्दों में क्या कहूँ, आपने मेरे लिए सब कुछ किया है। आप मेरी बुद्धि, मेरी ताकत, मेरा आशीर्वाद और मेरी प्रेरणा हैं। आपका प्यार मुझे हर दिन बढ़ने और आगे बढ़ने में मदद करता है। आपके बिना मैं कुछ नहीं हूँ।

प्रश्न: चाय बनाने की विधि बताइए।

उत्तर: पानी के बर्तन में पानी उबालें और चाय के पत्ते डाल दें। 5 से 7 मिनट तक उबलने दें, फिर चाय को छान लें। एक कप या चम्मच चीनी या मिश्रण जोड़ें और चाय का आनंद लें।

प्रश्न: भारत की राजधानी क्या है? एक वाक्य में उत्तर दीजिए।

उत्तर: भारत की राजधानी नई दिल्ली है।

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("pankajpandey-dev/qwen3.5-9b-hindi-instruct", torch_dtype="bfloat16", device_map="auto")
t = AutoTokenizer.from_pretrained("pankajpandey-dev/qwen3.5-9b-hindi-instruct")
msgs = [{"role": "user", "content": "जल संरक्षण के पाँच तरीके बताइए।"}]
text = t.apply_chat_template(msgs, add_generation_prompt=True, tokenize=False)
out = m.generate(**t(text=text, return_tensors="pt").to(m.device),
                 max_new_tokens=400, repetition_penalty=1.1)
print(t.decode(out[0], skip_special_tokens=True))

Use repetition_penalty=1.1 (set in this repo's generation_config) — long letter/essay outputs can loop without it. The chat template opens a think block; this model closes it immediately — strip <think>...</think> from output if present.

Training recipe (fully reproducible)


Base	unsloth/Qwen3.5-9B (bf16)
Method	LoRA r=16, alpha=16, response-only loss (Unsloth)
Data	12,912 Hindi pairs — anudesh 5,000 · dolly-hi 4,000 (chrF++ >= 55) · wikiHow-hi 3,000 · Aya-hi 912
Schedule	2 epochs, LR 1e-4 cosine, effective batch 16, seq 2048
Hardware	1x NVIDIA L40S (48 GB), ~135 min
Final train loss	0.938

Data deduplicated across sources, filtered for length and Latin-script ratio. wikiHow adds long-form step-by-step answers; Aya adds human-written originals.

Limitations

Parts of the data are machine-translated (dolly) or model-generated (anudesh via Llama-2-70B), so occasional unnatural phrasing or factual errors occur. Long formal-letter outputs may repeat without repetition_penalty=1.1. No additional safety tuning. Knowledge cutoff follows base Qwen3.5.

Licensing

Weights: Apache 2.0. Data licenses apply to data, not weights: dolly-hi CC-BY-SA 3.0-derived; anudesh generated by Llama-2-70B (Llama 2 license); Aya Apache 2.0.

🇮🇳 About the Hindi LLM Series

Weekly open releases making small LLMs speak fluent, native Hindi — trained on free/low-cost GPUs, shipped as GGUF for laptops and edge devices. Built by pankajpandey-dev (contact links on profile).

This release: Model · GGUF · LoRA · Series: 🇮🇳 Hindi LLM Collection