Truncating Response
#52
by
Mostafaadel174
- opened
I am experiencing issues with truncating responses. Where the generated response is cut off and doesn't complete the sentence
@Mostafaadel174 - Did you find any workaround?
I don't think it is honoring "max_new_tokens" parameter. Most of the responses are truncating below ~230 tokens (which includes the prompt itself). Any suggestions on how to fix it?