Is the hugginface implementation broken?
I can't get anything but some gibberish out of this model using the sample code, has anyone managed to make it work?
example:
<|START_OF_TURN_TOKEN|><|USER_TOKEN|>write a python script to send an http request<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>-sectionauthor
sectionauthor
sectionauthor
sectionauthor-sectionauthor-sectionauthor-sectionauthor
sectionauthor
sectionauthor
sectionauthor
sectionauthor
sectionauthor-sectionauthor-sectionauthor
begin-sectionauthor-sectionauthor-section-sectionauthor-sectionauthor-sectionauthor-sectionauthor
author
begin-begin-begin-begin-begin-begin-begin-begin-sectionauthor-begin-begin
author-begin-begin-begin-author-begin-begin-begin-author-begin-begin-author-author-author-author-author-author-author-author-begin-author-begin-author-author-author-author-author-author-author-author-author-begin-author-author-author-author-author-author-author-author-author-author-n-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author-author--author-author--author---author--author--author-author-----author--author--------------
I'm getting a similar response using HuggingFace TGI with this model as well.
hi, this might be due to your transformers version. For this model, you should install transformers from the source repo (latest commit) using pip install 'git+https://github.com/huggingface/transformers.git' as it includes modifications required for this model.
Thanks that was the problem