souvick-custom-qwen-coder

A fine-tuned and merged checkpoint of Qwen2.5-Coder-1.5B adapted using Low-Rank Adaptation (LoRA) for lightweight DevOps and scripting tasks. This model has had its LoRA adapter weights merged directly into the base model parameters, allowing standalone usage with standard transformers pipelines without extra PEFT loading steps.

Model Details

Model Description

• Developed by: Souvick Singha Roy
• Model Type: Causal Language Model (Code & Text Generation)
• Base Architecture: Qwen2 (Qwen2ForCausalLM)
• Base Model: Qwen/Qwen2.5-Coder-1.5B
• Language(s): English, Python, Bash, Shell
• License: Apache 2.0 (inherited from Qwen2.5)
• Repository: souvroy20/souvick-custom-qwen-coder

Architectural Specifications

• Parameters: ~1.54 Billion
• Context Window: 32,768 tokens (max_position_embeddings)
• Vocabulary Size: 151,936 tokens
• Format: Merged standalone model (model.safetensors, fp32 weights)

---

Intended Uses & Limitations

Direct Use

• Generating bash scripts, DevOps automation snippets, and general Python utility functions.
• Local experimentation and lightweight CPU inference using Transformers or converted GGUF formats (llama.cpp / Ollama).

Known Limitations & Generation Behavior

• Base Model Foundation: This model is fine-tuned on the Base version of Qwen2.5-Coder-1.5B, not the Instruct chat variant.
• Document Completion Tendency: Because it lacks extensive conversational RLHF alignment, the model behaves as a code and document completer. Without explicit stopping constraints, it may attempt to autocomplete multi-turn dialogues or emit dataset delimiters like <|repo_name|>.
• Recommended Inference Setup: Always pass stop_strings (e.g., ["<|im_end|>", "<|repo_name|>", "<|file_sep|>", "\nuser\n"]) and set a repetition_penalty of 1.15 during inference to ensure clean termination.

---

How to Get Started with the Model

1. Python Inference with Clean Stopping Logic

import time
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "souvroy20/souvick-custom-qwen-coder"

tokenizer = AutoTokenizer.from_pretrained(model_id)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token_id = tokenizer.eos_token_id

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.float32,
    device_map="cpu"
)

prompt = "Write a bash script to check free disk space."
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer([formatted_prompt], return_tensors="pt")
prompt_token_count = inputs.input_ids.shape[1]

# Stop strings prevent the base model from autocompleting extra turns or repo tags
stop_words = ["<|im_end|>", "<|repo_name|>", "<|file_sep|>", "\nuser\n", "\nAssistant:"]

start_time = time.perf_counter()
outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    do_sample=False,
    tokenizer=tokenizer,
    stop_strings=stop_words,
    pad_token_id=tokenizer.pad_token_id,
    repetition_penalty=1.15
)
elapsed = time.perf_counter() - start_time

generated_tokens = outputs[0][prompt_token_count:]
response = tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()

print("Generated Output:\n", response)
print(f"\nThroughput: {len(generated_tokens) / elapsed:.2f} tokens/s")