README.md

---
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
tags:
- text-generation-inference
- transformers
- qwen2
- trl
license: apache-2.0
language:
- en
---

# My first reasoning model inspire by **KingNish**:

```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

MAX_REASONING_TOKENS = 4096
MAX_RESPONSE_TOKENS = 256 # Recommend 512, 1024 Recommen


model = AutoModelForCausalLM.from_pretrained(
    'beyoru/ThinkAgain_1',
     torch_dtype=torch.float16,
     device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained('beyoru/ThinkAgain_1')

while True:
    prompt = input("USER: ")

    messages = [
        {"role": "user", "content": prompt}
    ]

    # Generate reasoning
    reasoning_template = tokenizer.apply_chat_template(messages, tokenize=False, add_reasoning_prompt=True)
    reasoning_inputs = tokenizer(reasoning_template, return_tensors="pt").to(model.device)
    reasoning_ids = model.generate(**reasoning_inputs, max_new_tokens=MAX_REASONING_TOKENS)
    reasoning_output = tokenizer.decode(reasoning_ids[0, reasoning_inputs.input_ids.shape[1]:], skip_special_tokens=True)

    print(reasoning_output)

    # Generate answer
    messages.append({"role": "reasoning", "content": reasoning_output})
    response_template = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    response_inputs = tokenizer(response_template, return_tensors="pt").to(model.device)
    response_ids = model.generate(**response_inputs, max_new_tokens=MAX_RESPONSE_TOKENS)
    response_output = tokenizer.decode(response_ids[0, response_inputs.input_ids.shape[1]:], skip_special_tokens=True)
    messages.append({"role": "assistant", "content": response_output})
    print(response_output)

```

Training on 2 hour with LoRA only attns layers

rank = 32, aplpha = 64
lr = 2e-4,

For the instruction task recommend to add in the user prompt, not the system prompt. Example:
Create SQL query for sepectific table you will provide:\
```<your question> \n <your table> \n <your desc if your table is complex>```



<img src="https://cdn-uploads.huggingface.co/production/uploads/65905af887944e494e37e09a/iOjhuLT0I2tT9Q0x2in2X.png" width="600">


# Weakness:
- Model still priority for English