OdiaGenAI/odiagenAI-model-v0

Model Card for Model ID

Model description

odiagenAI-model-v0 is based on Llama-7b and finetuned with 52k Odia translated data from the open-source Stanford-Alpaca, resulting in good Odia instruction understanding and response generation capabilities.

The code of Odia data generation and other detailed information can be found in our Github project repository: https://github.com/shantipriyap/OdiaGenAI. This repo contains a low-rank adapter for LLaMA-7b fit on the Stanford Alpaca dataset.

Training hyper-parameters

Parameter	Value
Batch size	128
Learning rate	3e-4
Epochs	2
Cutoff length	256
Weight_decay	0.001
Warmup_rate	0.1
LR_scheduler	linear
Lora r	16
Lora target modules	(q_proj, k_proj, v_proj, o_proj)

Model can be easily loaded with AutoModelForCausalLM.

import torch
from peft import PeftModel
import transformers

assert (
    "LlamaTokenizer" in transformers._import_structure["models.llama"]
), "LLaMA is now in HuggingFace's main branch.\nPlease reinstall it: pip uninstall transformers && pip install git+https://github.com/huggingface/transformers.git"
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig

tokenizer = LlamaTokenizer.from_pretrained("decapoda-research/llama-7b-hf")

BASE_MODEL = "decapoda-research/llama-7b-hf"
LORA_WEIGHTS = "OdiaGenAI/odiagenAI-model-v0"


model = LlamaForCausalLM.from_pretrained(
        BASE_MODEL,
        load_in_8bit=False,
        torch_dtype=torch.float16,
        device_map="auto",
    )
model = PeftModel.from_pretrained(
        model, LORA_WEIGHTS, torch_dtype=torch.float16, force_download=True
    )

def generate_prompt(instruction, input=None):
    if input:
        return f"""ନିମ୍ନରେ ଏକ ନିର୍ଦ୍ଦେଶନାମା ଯାହାକି ଏକ କାର୍ଯ୍ୟକୁ ବର୍ଣ୍ଣନା କରେ, ଏକ ଇନପୁଟ୍ ସହିତ ଯୋଡି ଯାହା ପରବର୍ତ୍ତୀ ପ୍ରସଙ୍ଗ ପ୍ରଦାନ କରେ | ଏକ ପ୍ରତିକ୍ରିୟା ଲେଖନ୍ତୁ ଯାହା ଅନୁରୋଧକୁ ସଠିକ୍ ଭାବରେ ସମାପ୍ତ କରେ |
### ନିର୍ଦ୍ଦେଶ:
{instruction}
### ଇନପୁଟ୍:
{input}
### ପ୍ରତିକ୍ରିୟା:"""
    else:
        return f"""ନିମ୍ନରେ ଏକ ନିର୍ଦ୍ଦେଶ ଯାହାକି ଏକ କାର୍ଯ୍ୟକୁ ବର୍ଣ୍ଣନା କରେ | ଏକ ପ୍ରତିକ୍ରିୟା ଲେଖନ୍ତୁ ଯାହା ଅନୁରୋଧକୁ ସଠିକ୍ ଭାବରେ ସମାପ୍ତ କରେ |
### ନିର୍ଦ୍ଦେଶ:
{instruction}
### ପ୍ରତିକ୍ରିୟା:"""

prompt = generate_prompt(instruction, input)
inputs = tokenizer(prompt, return_tensors="pt")
input_ids = inputs["input_ids"].to(device)
generation_config = GenerationConfig(
    temperature=0.1,
    top_p=0.75,
    top_k=40,
    num_beams=4,
    **kwargs,
)
with torch.no_grad():
    generation_output = model.generate(
        input_ids=input_ids,
        generation_config=generation_config,
        return_dict_in_generate=True,
        output_scores=True,
        max_new_tokens=128,
    )
s = generation_output.sequences[0]
output = tokenizer.decode(s)
print(output.split("### Response:")[1].strip())

Instructions for running it can be found at https://github.com/shantipriyap/OdiaGenAI.