Text2Text Generation
Transformers
GGUF
101 languages
t5
text-generation-inference
aya-101-GGUF / README.md
kcoopermiller's picture
Update README.md
90c93fb verified
---
license: apache-2.0
inference: false
datasets:
- CohereForAI/xP3x
- CohereForAI/aya_dataset
- CohereForAI/aya_collection
- DataProvenanceInitiative/Commercially-Verified-Licenses
- CohereForAI/aya_evaluation_suite
language:
- afr
- amh
- ara
- aze
- bel
- ben
- bul
- cat
- ceb
- ces
- cym
- dan
- deu
- ell
- eng
- epo
- est
- eus
- fin
- fil
- fra
- fry
- gla
- gle
- glg
- guj
- hat
- hau
- heb
- hin
- hun
- hye
- ibo
- ind
- isl
- ita
- jav
- jpn
- kan
- kat
- kaz
- khm
- kir
- kor
- kur
- lao
- lav
- lat
- lit
- ltz
- mal
- mar
- mkd
- mlg
- mlt
- mon
- mri
- msa
- mya
- nep
- nld
- nor
- nso
- nya
- ory
- pan
- pes
- pol
- por
- pus
- ron
- rus
- sin
- slk
- slv
- smo
- sna
- snd
- som
- sot
- spa
- sqi
- srp
- sun
- swa
- swe
- tam
- tel
- tgk
- tha
- tur
- twi
- ukr
- urd
- uzb
- vie
- xho
- yid
- yor
- zho
- zul
metrics:
- accuracy
- bleu
---
# Aya-101-GGUF
This repo contains GGUF format model files for Cohere's [Aya-101](https://huggingface.co/CohereForAI/aya-101) model
Quantized using Huggingface's [candle](https://github.com/huggingface/candle) framework
## How to use with Candle's quantized T5 example
Visit the [candle T5 example](https://github.com/huggingface/candle/tree/main/candle-examples/examples/quantized-t5) for more detailed instruction
1. Clone candle repo:
```bash
git clone https://github.com/huggingface/candle.git
cd candle/candle-examples
```
2. Run the following command:
```bash
cargo run --example quantized-t5 --release -- \
--model-id "kcoopermiller/aya-101-GGUF" \
--weight-file "aya-101.Q2_K.gguf" \
--config-file "config.json" \
--prompt "भारत में इतनी सारी भाषाएँ क्यों हैं?" \
--temperature 0
```
Available weight files:
- aya-101.Q2_K.gguf
- aya-101.Q3_K.gguf
- aya-101.Q4_0.gguf
- aya-101.Q4_1.gguf
- aya-101.Q4_K.gguf
- aya-101.Q5_0.gguf
- aya-101.Q5_1.gguf
- aya-101.Q5_K.gguf
- aya-101.Q6_K.gguf
- aya-101.Q8_0.gguf
- aya-101.Q8_1.gguf (not supported on candle yet)
- aya-101.Q8_K.gguf (not supported on candle yet)