update tokenizer for compatibility with new transformers (#64)
Browse files- update tokenizer for compatibility with new transformers (bbb5eacef5efb6b1d0b75c89296f576417bed4e1)
Co-authored-by: Ekaterina Aidova <katuni4ka@users.noreply.huggingface.co>