---
library_name: peft
base_model: unsloth/tinyllama-bnb-4bit
license: mit
datasets:
- yahma/alpaca-cleaned
language:
- en
pipeline_tag: text-generation
tags:
- Instruct
- TinyLlama
---

# Steps to try the model:

### prompt Template 
```python
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""
```
### load the model

```python
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM ,AutoTokenizer

config = PeftConfig.from_pretrained("damerajee/Tinyllama-sft-small")
model = AutoModelForCausalLM.from_pretrained("unsloth/tinyllama")
tokenizer=AutoTokenizer.from_pretrained("damerajee/Tinyllama-sft-small")
model = PeftModel.from_pretrained(model, "damerajee/Tinyllama-sft-small")l")

```
### Inference

```python
inputs = tokenizer(
[
    alpaca_prompt.format(
        "i want to learn machine learning help me", 
        "", # input
        "", # output
    )
]*1, return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 312, use_cache = True)
tokenizer.batch_decode(outputs)
```

# Model Information
The base model [unsloth/tinyllama-bnb-4bit](https://huggingface.co/unsloth/tinyllama-bnb-4bit)was Instruct finetuned using [Unsloth](https://github.com/unslothai/unsloth)

# Training Details

The model was trained for 1 epoch on a free goggle colab which took about 1 hour and 30 mins approximately