TGI Support

#16
by Khalidnass - opened

Could you please provide information on whether Mistralai/Mixtral-8x22B-Instruct-v0.1 will support TGI?

It is supported on TGI v2.0.1, I'm using it right now :)

sophiamyang changed discussion status to closed

What are the values that you used for: --max-input-length= --max-batch-prefill-tokens= --max-total-tokens=

Sign up or log in to comment