Best practices for empty or blank strings?
#3
by
grofte
- opened
The model fails for me if I pass an empty string, "", or a blank string, " ", because of something with tokenization.
What are the best practices for these cases? I just replaced with an x but I don't know if there are other ways I should do it.
And I feel like I didn't have this problem before. Did the model change?
Hi! This is a bug in the transformers package, which has been fixed in the newest development version (i.e., hasn't been released officially yet).
Try running pip install git+https://github.com/saattrupdan/transformers and see if that fixed the error :)
Closing this now, but feel free to re-open if it doesn't work!
saattrupdan
changed discussion status to
closed