README.md

---
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
base_model: unsloth/llama-3-8b-Instruct-bnb-4bit
---

# Model: BAI_LLM_FinArg

- **Developed by:** varadsrivastava
- **License:** apache-2.0
- **Base Model :** unsloth/llama-3-8b-Instruct-bnb-4bit

# For Proper Inference, please use:
!pip install "unsloth[colab-new] @ git+https://GitHub.com/unslothai/unsloth.git

### Loading the fine-tuned model and the tokenizer for inference
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
        model_name = "varadsrivastava/BAI_LLM_FinArg",
        max_seq_length = 20,
        dtype = torch.bfloat16,
        load_in_4bit = True
    )

### Using FastLanguageModel for fast inference
FastLanguageModel.for_inference(model)

# Prompt template:
"""<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{instruction}<|eot_id|><|start_header_id|>user<|end_header_id|>

Sentence: {row['text']}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Class: {row['label']}<|eot_id|>"""

NOTE: This model was trained 2x faster using Unsloth and Huggingface's TRL library.