Model Card for Model ID

Model description

odia_llama2_512 is based on Llama2-7b and finetuned with 180k Odia instruction set. The instruction set is translated data from open-source resources and a prepared domain knowledge instruction set, resulting in good Odia instruction understanding and response generation capabilities.

The code of Odia data generation and other detailed information can be found in our Github project repository: https://github.com/OdiaGenAI/GenerativeAI_and_LLM_Odia.

Training procedure

The following bitsandbytes quantization config was used during training:

quant_method: bitsandbytes
load_in_8bit: False
load_in_4bit: True
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: False
bnb_4bit_compute_dtype: float16

Framework versions

PEFT 0.5.0

Instructions for running it can be found at https://github.com/OdiaGenAI/GenerativeAI_and_LLM_Odia.

Licensing Information

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Citation Information

If you find this helpful repository, please consider giving 👏 and citing:

@misc{OdiaGenAI-Odia_Llama2_512,
  author = {Shantipriya Parida and Sambit Sekhar and Guneet Singh Kohli},
  title = {OdiaGenAI-Odia_Llama2_512},
  year = {2023},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  howpublished = {\url{https://huggingface.co/OdiaGenAI}},
}

Contributors:

Guneet Singh Kohli
Sambit Shekhar
Shantipriya Parida