Copy the tokenizer.json from the onnx/ to the root folder

#10
by libooo - opened

Copy the tokenizer.json into the root folder, so that toolings (e.g. https://github.com/elixir-nx/bumblebee#tokenizer-support) can find and use the fast tokenizer.

Thanks for fixing this @libooo !

oliverguhr changed pull request status to merged

Sign up or log in to comment