File size: 662 Bytes
bac7ce0 f154c90 bac7ce0 f154c90 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
---
language: en
license: apache-2.0
---
### Description
Adaptation of the [flan-t5-large](https://huggingface.co/google/flan-t5-large) weights to make it compatible with the [FAT5](https://github.com/catie-aq/flashT5) framework (Flash Attention T5).
This adaptation should enable the user to efficiently continue the pre-training of the flan-t5 to adapt it to more recent data, or to specialize it in a specific domain, for example.
### Usage
```
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("CATIE-AQ/FAT5-large-flan-en", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large")
``` |