Instrct version models keep missing the first few letters of the answer
I just tried the Inference API on the model page, and I noticed that both 2b/7b-it appear to miss the first few letters in their answers, why is that?
example:
<start_of_turn>user
Who is Steve Jobs?<end_of_turn>
<start_of_turn>model
ure, Steve Jobs was an American entrepreneur and businessman who co-founded Apple Inc. in 1976 and revolutionized the personal computer industry with the Apple Mac and the iPhone. He was born in 1955 and passed away in 2011.
<start_of_turn>user
Then who are the other co-founders of Apple?<end_of_turn>
<start_of_turn>model
ure, Steve Wozniak and John Sculley were the other co-founders of Apple Inc.
hmm, I have not seen this before, or in others' issues, are you doing any extra postprocessing by chance?
cc @sbrandeis as well, who is working on it :)
hmm, I have not seen this before, or in others' issues, are you doing any extra postprocessing by chance?
No. Actually it is reproducible I assume. I tried again it still skip the "S" in "Sure".
Acutally with the default example "how does the brain work" it also happened. The first "*" was missing.
<start_of_turn>user
How does the brain work?<end_of_turn>
<start_of_turn>model
*The brain is a complex organ that plays a vital role in many aspects of human cognition and behavior.** It is made up of billions of neurons, which are specialized cells that communicate with each other through electrical and chemical signals.
This is most likely due to a prefix space NOT being added/generated, but the decoding still removing the (non-existent) prefix space. Either way, this should be fixed with the updated version of the conversational widget. Thanks for reporting!
Can you confirm the problem is fixed @cooldog ?
Can you confirm the problem is fixed @cooldog ?
Hi, was any change made to the widget? I just tried it, and it seems a bit weird.
if I dont add the line breaker at the end of model, it returned as below: no line breaker added automatically, but the word "Sure" was complete.
If I add a line breaker, it returned as below: extra breaker added, and "Sure" missing the first two letters.
I assume this can be reproduced on your browser too?
I don't see this issue locally, but I have a difficult time clearing the example box, @osanseviero is there a fix here incoming?