tokenizer_config.json · oddadmix/bert-arabic-with-preprocessor at 957cddd79fcb4598c3ca4f82e0af0a9cb59e18d9

bert-arabic-with-preprocessor / tokenizer_config.json

Ahmed Wasfy

First model version

3bf56d2 8 months ago

81 Bytes

{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0}