|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
Flores101 model converted to HF format |
|
|
|
|
|
## Example |
|
Usage is the same as M2M100 exept the need to fix tokenizer. |
|
|
|
``` |
|
tokenizer.lang_token_to_id = {t: i for t, i in zip(tokenizer.all_special_tokens, tokenizer.all_special_ids) if i > 5} |
|
tokenizer.lang_code_to_token = {s.strip("_"): s for s in tokenizer.lang_token_to_id} |
|
tokenizer.lang_code_to_id = {s.strip("_"): i for s, i in tokenizer.lang_token_to_id.items()} |
|
tokenizer.id_to_lang_token = {i: s for s, i in tokenizer.lang_token_to_id.items()} |
|
``` |
