Getting issue in model text generation in different language
This is my prompt
def stream(user_prompt):
system_prompt='You are a helpful assistant that provides accuracte and concise responses to the user'
B_INST,E_INST="[INST]","[/INST]"
B_SYS,E_SYS="\n","\n<>\n\n"
prompt=f"{B_INST}{B_SYS}{system_prompt.strip()}{E_SYS}{user_prompt.strip()}{E_INST}\n\n"
inputs=tokenizer([prompt],return_tensors="pt").to(runtimeFlag)
streamer=TextStreamer(tokenizer)
_=model.generate(**inputs,streamer=streamer,max_new_tokens=500)
stream("Could you please provide information about the internet in English?")
and this is the output
Of фев Hinweis surely Hinweis nobody Unterscheidung nobody ultimately ultimately surely фев фев Hinweis brings Hinweis лютого surely hopefully nobody
I can't see your B_SYS and E_SYS, you have to wrap them in back ticks to avoid markdown eating the contents.
Also, try using the tokenizer.apply_chat_template, as recommended on the card, for formatting the messages.
what changes do i need here?
def stream(user_prompt):
system_prompt='You are a helpful assistant that provides accuracte and concise responses to the user'
B_INST,E_INST="[INST]","[/INST]"
B_SYS,E_SYS="\n","\n<>\n\n"
prompt=f"{B_INST}{B_SYS}{system_prompt.strip()}{E_SYS}{user_prompt.strip()}{E_INST}\n\n"
inputs=tokenizer([prompt],return_tensors="pt").to(runtimeFlag)
streamer=TextStreamer(tokenizer)
_=model.generate(**inputs,streamer=streamer,max_new_tokens=500)
Function metadata is missing.
Please try using the recommended apply_chat_template approach.