Text Generation
Transformers
PyTorch
Indonesian
English
llama
text-generation-inference
Inference Endpoints
File size: 7,487 Bytes
cf8688a
 
 
98beccf
 
cf8688a
 
 
 
5e75a16
cf8688a
0831a22
 
 
cf8688a
1ecc53c
cf8688a
1860717
cf8688a
 
 
 
 
ca04fde
cf8688a
 
 
1ecc53c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5dcaa0f
1ecc53c
 
 
 
 
3a09cf9
1ecc53c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5dcaa0f
1ecc53c
 
 
 
 
3a09cf9
1ecc53c
 
 
 
 
5bc1a6a
bbe2ef1
 
5bc1a6a
 
cf8688a
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
dbee275
 
 
 
 
 
 
 
 
 
 
 
 
cf8688a
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
---
datasets:
- wikipedia
- Ichsan2895/OASST_Top1_Indonesian
- Ichsan2895/alpaca-gpt4-indonesian
language:
- id
- en
pipeline_tag: text-generation
license: cc-by-nc-sa-4.0
---
<div style="width: auto; margin-left: auto; margin-right: auto">
<img src="https://huggingface.co/Ichsan2895/Merak-7B-v3/resolve/main/FINAL_LOGO/6.png" alt="MERAK" style="width: 50%; min-width: 100px; display: block; margin: auto;">
</div>

# HAPPY TO ANNOUNCE THE RELEASE OF MERAK-7B-V3!

Merak-7B is the Large Language Model of Indonesian Language 

This model is based on Meta Llama-2-7B-Chat-HF and fine tuned by some of Indonesia Wikipedia articles that I cleaned before.

Leveraging QLoRA (QLora: Efficient Finetuning of Quantized LLMs), Merak-7B is able to run with 16 GB VRAM

Merak-7B and all of its derivatives are Licensed under Creative Commons-By Attribution-Share Alike-Non Commercial (CC-BY-SA-NC 4.0). Merak-7B empowers AI enthusiasts, researchers alike.

Big thanks to all my friends and communities that help to build our first model. Feel free, to ask me about the model and please share the news on your social media.

## HOW TO USE
### Installation
Please make sure you have installed CUDA driver in your system, Python 3.10 and PyTorch 2. Then install this library in terminal
```
pip install bitsandbytes==0.39.1
pip install transformers==4.31.0
pip install peft==0.4.0
pip install accelerate==0.20.3
pip install einops==0.6.1 scipy sentencepiece datasets
```
### Using BitsandBytes and it run with >= 10 GB VRAM GPU
[![Open in Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1gCCHo2KvqLr8Sf6aIbE9NLkOpn_w4v94?usp=drive_link)
```
import torch
from transformers import AutoTokenizer, AutoConfig, AutoModelForCausalLM, BitsAndBytesConfig, LlamaTokenizer
from peft import PeftModel, PeftConfig

model_id = "Ichsan2895/Merak-7B-v3"
config = AutoConfig.from_pretrained(model_id)

BNB_CONFIG = BitsAndBytesConfig(load_in_4bit=True,
                                bnb_4bit_compute_dtype=torch.bfloat16,
                                bnb_4bit_use_double_quant=True,
                                bnb_4bit_quant_type="nf4",
    )

model = AutoModelForCausalLM.from_pretrained(model_id,
                                             quantization_config=BNB_CONFIG,
                                             device_map="auto",
                                             trust_remote_code=True)

tokenizer = LlamaTokenizer.from_pretrained(model_id)

def generate_response(question: str) -> str:
  prompt = f"<|prompt|>{question}\n<|answer|>".strip()

  encoding = tokenizer(prompt, return_tensors='pt').to("cuda")
  with torch.inference_mode():
    outputs = model.generate(input_ids=encoding.input_ids,
                             attention_mask=encoding.attention_mask,
                             eos_token_id=tokenizer.pad_token_id,
                             do_sample=False,
                             num_beams=2,
                             temperature=0.3,
                             repetition_penalty=1.2,
                             max_new_tokens=2048)
    
    response = tokenizer.decode(outputs[0], skip_special_tokes=True)

    assistant_start = "<|answer|>"
    response_start = response.find(assistant_start)
    return response[response_start + len(assistant_start) :].strip()

prompt = "Siapa penulis naskah proklamasi kemerdekaan Indonesia?"
print(generate_response(prompt))
```


### From my experience, For better answer, please don’t use BitsandBytes 4-bit Quantization, but it using higher VRAM
[![Open in Google Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1rOeFT9cC2OzlW6CUoEe4rpXn2USI3l-E?usp=drive_link)
```
import torch
from transformers import AutoTokenizer, AutoConfig, AutoModelForCausalLM, BitsAndBytesConfig, LlamaTokenizer
from peft import PeftModel, PeftConfig

model_id = "Ichsan2895/Merak-7B-v3"
config = AutoConfig.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id,
                                             device_map="auto",
                                             trust_remote_code=True)

tokenizer = LlamaTokenizer.from_pretrained(model_id)

def generate_response(question: str) -> str:
  prompt = f"<|prompt|>{question}\n<|answer|>".strip()

  encoding = tokenizer(prompt, return_tensors='pt').to("cuda")
  with torch.inference_mode():
    outputs = model.generate(input_ids=encoding.input_ids,
                             attention_mask=encoding.attention_mask,
                             eos_token_id=tokenizer.pad_token_id,
                             do_sample=False,
                             num_beams=2,
                             temperature=0.3,
                             repetition_penalty=1.2,
                             max_new_tokens=2048)
    
    response = tokenizer.decode(outputs[0], skip_special_tokes=True)

    assistant_start = "<|answer|>"
    response_start = response.find(assistant_start)
    return response[response_start + len(assistant_start) :].strip()

prompt = "Siapa penulis naskah proklamasi kemerdekaan Indonesia?"
print(generate_response(prompt))
```

## CHANGELOG
**v3** = Fine tuned by [Ichsan2895/OASST_Top1_Indonesian](https://huggingface.co/datasets/Ichsan2895/OASST_Top1_Indonesian) & [Ichsan2895/alpaca-gpt4-indonesian](https://huggingface.co/datasets/Ichsan2895/alpaca-gpt4-indonesian)  
**v2** = Finetuned version of first Merak-7B model. We finetuned again with the same ID Wikipedia articles except it changes prompt-style in the questions. It has 600k ID wikipedia articles.  
**v1** = The first Merak-7B model. We selected and cleaned about 200k ID wikipedia articles.  

## CITATION
```
@Paper{arXiv,
  author  = {Touvron, et al},
  title   = {Llama 2: Open Foundation and Fine-Tuned Chat Models},
  journal = {arXiv preprint arXiv:2307.09288},
  year    = {2023}
}

@ONLINE{wikidump,
    author = "Wikimedia Foundation",
    title  = "Wikimedia Downloads",
    url    = "https://dumps.wikimedia.org"
}

@inproceedings{wolf-etal-2020-transformers,
    title = "Transformers: State-of-the-Art Natural Language Processing",
    author = "Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and Rémi Louf and Morgan Funtowicz and Joe Davison and Sam Shleifer and Patrick von Platen and Clara Ma and Yacine Jernite and Julien Plu and Canwen Xu and Teven Le Scao and Sylvain Gugger and Mariama Drame and Quentin Lhoest and Alexander M. Rush",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = oct,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.emnlp-demos.6",
    pages = "38--45"
}

@article{dettmers2023qlora,
  title   = {QLoRA: Efficient Finetuning of Quantized LLMs},
  author  = {Dettmers, Tim and Pagnoni, Artidoro and Holtzman, Ari and Zettlemoyer, Luke},
  journal = {arXiv preprint arXiv:2305.14314},
  year    = {2023}
}
```

## HOW TO CITE THIS PROJECT

If you use the Merak-7B model in your research or project, please cite it as:
```
@article{Merak,
  title={Merak-7B: The LLM for Bahasa Indonesia},
  author={Muhammad Ichsan},
  publisher={Hugging Face}
  journal={Hugging Face Repository},
  year={2023}
}
```