can ggml be kept on main branch?
#2
by
matthoffner
- opened
I'm interested in getting the new format to work but I'd like to use the old format. I'm not sure how to do this in ctransformers for example, so I have to download the model manually.
matthoffner
changed discussion title from
can we keep ggml on main branch?
to can ggml be kept on main branch?
Ah yeah good point, ctransformers doesn't support branches/revisions yet. I have raised a request for @marella here: https://github.com/marella/ctransformers/issues/51
It should be really easy to implement so I'm sure he'll do it soon.
In the meantime you can use huggingface_hub.snapshot_download to download it, like so:
from huggingface_hub import snapshot_download
snapshot_download(repo_id="TheBloke/falcon-7b-instruct-GGML",
filename="falcon7b-instruct.ggmlv3.q4_0.bin",
revision="ggmlv3",
local_dir="/path/to/desired_folder/",
local_dir_use_symlinks=False)
Then use ctransformers to load the local file.
thanks!
matthoffner
changed discussion status to
closed