Deployed successfully on SageMaker, but the output is truncated.
#2
by
raafat1983
- opened
I manage to delpoy Yi-6B-Chat on Amazon SageMAker and successfully did inference , however but the output is truncated. How I can increase the output token
[# send request
predictor.predict({
"inputs": "give me a list of 10 city"
})]
[{'generated_text': 'give me a list of 10 city names in the world that start with the letter "C".Here is a list of 1'}]
Adjust the max_new_tokens parameter of GenerationConfig
lorinma
changed discussion status to
closed