remove unk_token : null value to prevent issues with downstream tools
#3
by
SOBAMD
- opened
Most tools ignore the null value but a few tools expect a string value or no value at all and this prevents them from loading the model correctly. The issue is specifically with transformers > 4.44.2 which has knock on effects into other tooling.