How to get longer outputs?
#49
by
Apps
- opened
Im using Inference Enpoints to do QA over docs:
Prompt:
CONTEXT:
<document chunk 1>
<document chunk 2>
<document chunk 3>
<document chunk 4>
<document chunk 5>
QUESTION: What is the answer to life, the universe and everything?
ANSWER:
I usually get very short outputs of one or two words. How can i get longer outputs?
Thanks
You can force a minimum generation length by setting the min_new_tokens kwarg to e.g. 100