|
--- |
|
license: cc-by-4.0 |
|
language: |
|
- he |
|
inference: false |
|
--- |
|
# **DictaLM**: A Large Generative Language Model for Modern Hebrew |
|
|
|
A large generative pretrained transformer (GPT) language model for Hebrew, released [link will be updated soon]. |
|
|
|
- This is an alpha version of the model, and there are many improvements to come. |
|
- We are actively working on improving the model, so stay tuned. |
|
|
|
|
|
This model was fine-tuned for instructions, here are a few examples of the different types of instructions the model was trained on: |
|
|
|
- General questions: |
|
``` |
|
ืื ืื ืืืช ืกืคืจ? |
|
``` |
|
|
|
``` |
|
ืงืืืืชื ืืชื ืงื ืืืฆืืข. ืืื ืืืจื ืื ืืื ื ืืืคื ืืื? |
|
``` |
|
- Simple tasks: |
|
``` |
|
ืชืฆืืข ืืื ืจืขืืื ืืช ืืคืขืืืืช ืขื ืืืืื ืื ื 5: |
|
``` |
|
- Information retrieval from a paragraph context: |
|
|
|
``` |
|
ืืืกืืง ืืืื ื ืืื ืืืจื ืืืกืืจืชืืช ืืืขืชืืงื ืืงืืืฃ ืืืชืื. ืฉืืื ืื ืืืจืฉืช ืืื ืืื ืจื ืืืืคื ืืืกื ืืขืืืื ืืงืืืืช ืืืฉืจืื ืืืืงืืืืช ืจืืื ืืขืืื. ืฉืืืืช ืืกืืง ืืื ื ืืืคืฉืจืืช ืืืกืืื ืขืืืืืช ืืืงืืืืช ืืื ืืื ืืืื ืืื ืืขืืืช ืืฉืืืืช ืืืืืื ืืช ืืืืื. ืืืืชืื ืืืืืขืืื ืืืืื (ืืืืืฉื, ืื ืืืื ืืืืชืื ืืฉืื) ืืชืืื ืืืชืจ ืืกืืง ืืื ื ืืืืื ืฉืืคืจื ืคืืืช ื ืคืืข ืืืืื ืืืกืืง ืืฉืืื ืื (ืคืืืขืืช ืืงืืืคืช ืืคืจื ืืืืชืื ืืฉืื ืคืืืช ืืฉืืขืืชืืืช). ืืื ืื ืืืขืืฃ ืืกืืง ืืื ื ืืืืืจืื ืืื ืืืืคืืืจืคืื ืืืงืืืืช ืื ืฆืคืืคืืช ืืขืฆืื ืื ืืืคืฉืจืื ืืืฉื ื ืืื ืืืืื ืืื ืื. ืืฉืืื ืืืื ืืช ืืืคืฉืจืช ืื ืืืกืืง ืขืฆืื ืฉืื ืื ืืืืขืืื ืฉืื ืื, ืืืชืื ืืงืฆื ืืืฉืืช ืืคืจื ืืืืขื ืืื ืขืฅ. |
|
|
|
ืขื ืืกืืก ืืคืกืงื ืืืืช, ืื ืืื ืืืชืจืื ืฉื ืืกืืง ืืื ื ืืืืื ืช ืงืฆื ืืืฉืืช ืืคืจื? |
|
``` |
|
|
|
## Sample usage: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
import torch |
|
|
|
tokenizer = AutoTokenizer.from_pretrained('dicta-il/dictalm-7b-instruct') |
|
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True).cuda() |
|
|
|
model.eval() |
|
|
|
with torch.inference_mode(): |
|
prompt = 'ืชืฆืืข ืืื ืจืขืืื ืืช ืืคืขืืืืช ืขื ืืืืื ืื ื 5:\n' |
|
kwargs = dict( |
|
inputs=tokenizer(prompt, return_tensors='pt').input_ids.to(model.device), |
|
do_sample=True, |
|
top_k=50, |
|
top_p=0.95, |
|
temperature=0.75, |
|
max_length=100, |
|
min_new_tokens=5 |
|
) |
|
|
|
print(tokenizer.batch_decode(model.generate(**kwargs), skip_special_tokens=True)) |
|
``` |
|
|
|
|
|
There are many different parameters you can input into `kwargs` for different results (greedy, beamsearch, different sampling configurations, longer/shorter respones, etc.). |
|
|
|
You can view the full list of parameters you can pass to the `generate` function [here](https://huggingface.co/docs/transformers/v4.33.0/en/main_classes/text_generation#transformers.GenerationMixin.generate). |
|
|
|
### Alternative ways to initialize the model: |
|
|
|
If you have multiple smaller GPUs, and the package `accelerate` is installed, you can initialize the model split across the devices: |
|
```python |
|
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True, device_map='auto') |
|
``` |
|
|
|
If you are running on linux and have the `bitsandbytes` package installed, you can initialize the model in 4/8 bit inference mode: |
|
```python |
|
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True, load_in_8bit=True) |
|
``` |
|
|
|
If you have [FlashAttention](https://github.com/Dao-AILab/flash-attention) installed in your environment, you can instruct the model to use the flash attention implementation (either V1 or V2, whichever is installed): |
|
```python |
|
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True, use_flash_attention=True) |
|
``` |
|
|
|
|
|
## Citation |
|
|
|
If you use DictaLM in your research, please cite `DictaLM - A Large Generative Language Model for Modern Hebrew` [forthcoming]. |
|
|
|
**BibTeX:** |
|
|
|
[BibText will be updated soon] |
|
|
|
## License |
|
|
|
Shield: [![CC BY 4.0][cc-by-shield]][cc-by] |
|
|
|
This work is licensed under a |
|
[Creative Commons Attribution 4.0 International License][cc-by]. |
|
|
|
[![CC BY 4.0][cc-by-image]][cc-by] |
|
|
|
[cc-by]: http://creativecommons.org/licenses/by/4.0/ |
|
[cc-by-image]: https://i.creativecommons.org/l/by/4.0/88x31.png |
|
[cc-by-shield]: https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg |