Sorjîn1.1-7B-Instruct

Sorjîn1.1-7B-Instruct is a Kurdish Kurmanji instruction-following model produced by applying the Sorjîn1-LoRA adapter directly onto Qwen/Qwen2.5-7B-Instruct and merging it into the weights.

It combines Kurdish language knowledge from continual pretraining with the instruction-following capability of Qwen2.5-7B-Instruct.

What changed from Sorjîn1-7B-Instruct

Sorjîn1.1 uses the same continual pretraining as Sorjîn1 (the same LoRA adapter, no new training). Only the way the Kurdish knowledge is combined with the instruct model is different.

Sorjîn1-7B-Instruct was built with a TIES merge that used Qwen2.5-7B-Instruct as the merge base. With that base, the Kurdish delta was measured against the instruct model instead of against Qwen2.5-7B, so the merge also removed part of the instruction tuning. The result wrote fluent Kurdish but often did not answer the question.

Sorjîn1.1 avoids the problem by adding the Kurdish delta (the LoRA adapter) straight onto the instruct model:

Qwen2.5-7B-Instruct + Sorjîn1-LoRA = Sorjîn1.1-7B-Instruct

The full instruction tuning and the full Kurdish delta are both kept, with nothing trimmed.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "muzaffercky/Sorjin1.1-7B-Instruct",
    torch_dtype="bfloat16",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("muzaffercky/Sorjin1.1-7B-Instruct")

messages = [{"role": "user", "content": "Tu çawa yî?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Build Details

Method: LoRA adapter merged into the instruct model (merge_and_unload from PEFT)
Instruct model: Qwen/Qwen2.5-7B-Instruct
Adapter: Sorjîn1-LoRA (rank 64, alpha 128), trained with QLoRA on Qwen/Qwen2.5-7B
Training data: kurdish-kurmanji-theses (389 academic theses)
Tokenizer and chat template: unchanged from Qwen2.5-7B-Instruct

Limitations

Kurdish language coverage is limited to formal academic style — trained on university theses
The adapter was trained on Qwen2.5-7B, not on the instruct model, so the two can interfere slightly
Not fine-tuned on Kurdish instruction pairs — answers to Kurdish questions can still be off-topic or factually wrong
May mix languages (Kurdish/English/Turkish) on topics not covered in training data
Not aligned with RLHF — use with appropriate safety considerations

Model Family

Model	Description
Sorjîn1-LoRA	Raw LoRA adapter weights
Sorjîn1-7B	Continually pretrained base model
Sorjîn1-7B-Instruct	TIES merge with Qwen2.5-7B-Instruct (superseded)
Sorjîn1.1-7B-Instruct (this model)	LoRA applied directly to Qwen2.5-7B-Instruct