README.md

---
license: cc-by-nc-4.0
language:
- tr
pipeline_tag: text-generation
---

# Model Card for Model ID


Gemma-2b fine-tuned with Turkish Instruction-Response pairs.


### Training Data

- Dataset size: ~75k


## Restrictions

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms
Please refer to the gemma use restrictions before start using the model.
https://ai.google.dev/gemma/terms#3.2-use

## Using model

```Python
import torch,re
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "erythropygia/Gemma2b-Turkish-Instruction"

model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
tokenizer = AutoTokenizer.from_pretrained(model_id, add_eos_token=True, padding_side="left")

def get_completion(query: str, model, tokenizer) -> str:
  device = "cuda:0"

  prompt_template = """
  <start_of_turn>user
  Alt satırdaki soruya cevap ver:\n
  {query}
  <end_of_turn>\n<start_of_turn>model
  """
  prompt = prompt_template.format(query=query)

  encodeds = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)

  model_inputs = encodeds.to(device)


  #max_new_tokens = 200, temperature = 0.9, repetition_penalty = 0.5,  disabled
  #num_return_sequences=1, max_length = 256,
  generated_ids = model.generate(**model_inputs, max_new_tokens = 256, do_sample=True, pad_token_id=tokenizer.eos_token_id)
  decoded = tokenizer.decode(generated_ids[0], skip_special_tokens=False)


  decoded = re.sub(r'<(end_of_turn|start_of_turn|eos|bos)>[^<]*$', '', decoded)

  decoded = re.sub(r'<(end_of_turn|start_of_turn|eos|bos)>', '', decoded)

  return decoded.strip()

result = get_completion(query="int türünde üç parametre alan ve bunların toplamını döndüren bir işlev oluşturun.", model=model, tokenizer=tokenizer)
print(result)
```

#### Training Hyperparameters

- **Epochs:** 1
- **MaxSteps:** 300
- **Context length:** 1024
- **LoRA Rank:** 32
- **LoRA Alpha:** 64
- **LoRA Dropout:** 0.05

#### Training Results
 **training_loss:** 2.077697410186132 (could be better when i will find enough GPU access :-) )