|
--- |
|
license: apache-2.0 |
|
tags: |
|
- merge |
|
- hindi |
|
- english |
|
- Llama2 |
|
- ai4bharat/Airavata |
|
- BhabhaAI/Gajendra-v0.1 |
|
model-index: |
|
- name: yuj-v1 |
|
results: |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: AI2 Reasoning Challenge (25-Shot) |
|
type: ai2_arc |
|
config: ARC-Challenge |
|
split: test |
|
args: |
|
num_few_shot: 25 |
|
metrics: |
|
- type: acc_norm |
|
value: 45.65 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: HellaSwag (10-Shot) |
|
type: hellaswag |
|
split: validation |
|
args: |
|
num_few_shot: 10 |
|
metrics: |
|
- type: acc_norm |
|
value: 70.1 |
|
name: normalized accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: MMLU (5-Shot) |
|
type: cais/mmlu |
|
config: all |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 43.78 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: TruthfulQA (0-shot) |
|
type: truthful_qa |
|
config: multiple_choice |
|
split: validation |
|
args: |
|
num_few_shot: 0 |
|
metrics: |
|
- type: mc2 |
|
value: 41.69 |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: Winogrande (5-shot) |
|
type: winogrande |
|
config: winogrande_xl |
|
split: validation |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 69.85 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
- task: |
|
type: text-generation |
|
name: Text Generation |
|
dataset: |
|
name: GSM8k (5-shot) |
|
type: gsm8k |
|
config: main |
|
split: test |
|
args: |
|
num_few_shot: 5 |
|
metrics: |
|
- type: acc |
|
value: 4.78 |
|
name: accuracy |
|
source: |
|
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=shuvom/yuj-v1 |
|
name: Open LLM Leaderboard |
|
--- |
|
|
|
# The Model yuj-v1: |
|
|
|
The yuj-v1 model is a blend of advanced models strategically crafted to enhance Hindi Language Models (LLMs) effectively and democratically. Its primary goals include catalyzing the development of Hindi and its communities, making significant contributions to linguistic knowledge. The term "yuj," from Sanskrit, signifies fundamental unity, highlighting the integration of sophisticated technologies to improve the language experience for users in the Hindi-speaking community. |
|
|
|
Official GGUF version: [shuvom/yuj-v1-GGUF](https://huggingface.co/shuvom/yuj-v1-GGUF) |
|
|
|
Below are the model which are leverage to build this yuj-v1: |
|
* [ai4bharat/Airavata](https://huggingface.co/ai4bharat/Airavata) |
|
* [BhabhaAI/Gajendra-v0.1](https://huggingface.co/BhabhaAI/Gajendra-v0.1) |
|
|
|
## ☄️Space to use it (yuj-v1 tryO): |
|
<a target="_blank" href="https://shuvom-yuj-v1-tryo.hf.space"> |
|
<img src="https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg" alt="Open in HuggingFace"/> |
|
</a> |
|
|
|
|
|
|
|
|
|
## 💻 Usage: |
|
|
|
First, you need to install some of below packages: |
|
|
|
1. Bits and bytes |
|
```python |
|
!pip install bitsandbytes |
|
``` |
|
2. Accelerate (to install the latest version) |
|
```python |
|
!pip install git+https://github.com/huggingface/accelerate.git |
|
``` |
|
3. Usage |
|
```python |
|
# Usage |
|
import torch |
|
|
|
# Load model directly |
|
from transformers import AutoTokenizer, AutoModelForCausalLM |
|
|
|
# load the model in 4-bit quantization |
|
tokenizer = AutoTokenizer.from_pretrained("shuvom/yuj-v1") |
|
model = AutoModelForCausalLM.from_pretrained("shuvom/yuj-v1",torch_dtype=torch.bfloat16,load_in_4bit=True) |
|
|
|
prompt = "युज शीर्ष द्विभाषी मॉडल में से एक है" |
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
|
# Generate |
|
generate_ids = model.generate(inputs.input_ids, max_length=65) |
|
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] |
|
``` |
|
4. Output |
|
```python |
|
युज शीर्ष द्विभाषी मॉडल में से एक है। यह एक उत्पादक मॉडल है जो एक साथ एक ट्रांसफॉर्मर और एक आत्म-ध्यान तंत्रिका नेटवर्क को जोड़ता है। यह एक ट्रांसफॉर्मर वास्तुकला का उपयोग करता है जो एक ट्रांसफॉर्मर मॉडल की तुलना में बहुत अधिक जटिल है। |
|
``` |
|
|
|
## 🧩 Configuration |
|
|
|
```yaml |
|
models: |
|
- model: sarvamai/OpenHathi-7B-Hi-v0.1-Base |
|
# no parameters necessary for base model |
|
- model: ai4bharat/Airavata |
|
parameters: |
|
density: 0.5 |
|
weight: 0.5 |
|
- model: BhabhaAI/Gajendra-v0.1 |
|
parameters: |
|
density: 0.5 |
|
weight: 0.3 |
|
merge_method: ties |
|
base_model: sarvamai/OpenHathi-7B-Hi-v0.1-Base |
|
parameters: |
|
normalize: true |
|
dtype: float16 |
|
``` |
|
|
|
|
|
|
|
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) |
|
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_shuvom__yuj-v1) |
|
|
|
| Metric |Value| |
|
|---------------------------------|----:| |
|
|Avg. |45.97| |
|
|AI2 Reasoning Challenge (25-Shot)|45.65| |
|
|HellaSwag (10-Shot) |70.10| |
|
|MMLU (5-Shot) |43.78| |
|
|TruthfulQA (0-shot) |41.69| |
|
|Winogrande (5-shot) |69.85| |
|
|GSM8k (5-shot) | 4.78| |
|
|
|
|