Can you upload also the tokenizer.json?

#11

by nmaroulis - opened Mar 23

Discussion

nmaroulis

Mar 23

•

edited Mar 23

I notice that there is no tokenizer.json and the framework i am using it needs it to load the model (Elixir#bumblebee).
Is anyway possible to generate it or uploaded?

ddh0

Mar 24

seconded

qeternity

Mar 28

@teknium any chance of this?

ddh0

Mar 28

I was unable to convert the model correctly without the tokenizer, the output contained tokens like etc. would be nice to have the proper tokenizer

teknium

NousResearch org Apr 2

I don't know how I would create that

teknium changed discussion status to closed Apr 2

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment