File size: 1,735 Bytes
24adf28 c860f43 24adf28 c860f43 24adf28 c860f43 d0ec2c1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
---
inference: false
license: openrail
language:
- it
---
# ExtremITA Camoscio 7 bilion parameters adapters: ExtremITLLaMA
This is ExtremITLLaMA, the adapters for the instruction-tuned Italian LLaMA model that participated in all the tasks of [EVALITA 2023](https://www.evalita.it/campaigns/evalita-2023/) winning 41% of tasks and achieving 64% of top-three positions.
It requires the base model from [sag-uniroma2/extremITA-Camoscio-7b](https://huggingface.co/sag-uniroma2/extremITA-Camoscio-7b).
# Usage
Checkout the github repository for more insights and codes: https://github.com/crux82/ExtremITA
```python
from peft import PeftModel
from transformers import LLaMATokenizer, LLaMAForCausalLM
import torch
tokenizer = LLaMATokenizer.from_pretrained("yahma/llama-7b-hf")
model = LlamaForCausalLM.from_pretrained(
"sag-uniroma2/extremITA-Camoscio-7b",
load_in_8bit=True,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(
model,
"sag-uniroma2/extremITA-Camoscio-7b-adapters",
torch_dtype=torch.float16,
device_map="auto",
)
```
# Citation
```
@inproceedings{hromei2023extremita,
author = {Claudiu Daniel Hromei and
Danilo Croce and
Valerio Basile and
Roberto Basili},
title = {ExtremITA at EVALITA 2023: Multi-Task Sustainable Scaling to Large Language Models at its Extreme},
booktitle = {Proceedings of the Eighth Evaluation Campaign of Natural Language
Processing and Speech Tools for Italian. Final Workshop (EVALITA 2023)},
publisher = {CEUR.org},
year = {2023},
month = {September},
address = {Parma, Italy}
}
``` |