Inference API
Hello,
I want to use the inference API of this model but I am not getting the documentation as how to pass the my query in the body along with the suitable parameters.
If someone can help in the format structure of how to build the Query Body along with the parameters it would be really helpful.
Thanks & Regards,
Shiv Kumar
Hey @Shivkumar27
Check this response here: https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1/discussions/125#65c481d27fe765199ae3add2
the structure of your query should be something like this (from the discussion mentioned above)
query({
"inputs": "[INST] You are a helpful chatbot assistant which provides answer based on the context given. Do not give any extra information. Do not give the context again in your response\nGenerate a concise and informative answer in less than 100 words for the given question\nSEARCH RESULT 1: The title is Rush, year is 2013, budget is 500000, earning is 300000, genere is action.\nQUESTION: What is the release date of rush?\n[/INST]",
"parameters": {
"return_full_text": False
}
})
Edit: The discussion above is for Mixtral 8x7B, but I tested it with this model and it works fine