Empty response when using generated curl statement from API playground

#16
by jwschrader - opened

Hi folks,
Not sure if this is the best place to ask this question. Please let me know if there is somewhere else. I'll also probably try your support/contact link.

When I use the playground with this model, it works fine.

I then copy the curl command that the playground generates and run that locally and it returns an empty text string for the answer. Here is a sample response. I've tried adjusting the temperature, top_p, top_k, max_tokens, stream_tokens and the prompt that I'm asking.

{"id":"83df4b8d3f94c39c-SEA","status":"finished","prompt":["[INST]  What is rain?  [/INST]  "],"model":"togethercomputer/Llama-2-7B-32K-Instruct","model_owner":"","num_returns":1,"args":{"model":"togethercomputer/Llama-2-7B-32K-Instruct","prompt":"[INST]  What is rain?  [/INST]  ","request_type":"language-model-inference","temperature":0.1,"top_p":0.1,"top_k":3,"max_tokens":1000,"stream_tokens":false,"stop":["[INST]","\n\n"],"negative_prompt":"","sessionKey":"906e58d5-91b6-4ec9-990e-315b47ec5184","update_at":"2023-12-31T03:00:29.376Z"},"subjobs":[],"output":{"usage":{"prompt_tokens":15,"completion_tokens":2,"total_tokens":17},"result_type":"language-model-inference","choices":[{"text":""}]}}%

This also happens when using the javascript code that is generated.

The generated curl command is working correctly for all other models that I have tried.

Thanks for any guidance,
Jeremy

Together org

issue was fixed in the Playground! Please try again

orangetin changed discussion status to closed

Sign up or log in to comment