dialect-ar-gpt-2021 / added_tokens.json
monsoon-nlp's picture
initial commit
2dfd242
{"[EGYPTIAN]": 64000, "[MSA]": 64001, "[LEVANTINE]": 64002, "[GULF]": 64003, "[MAGHREBI]": 64004}