Is the sqlcoder model architecture supported by vllm?
Is the sqlcoder model architecture supported by vllm?
https://docs.vllm.ai/en/latest/models/supported_models.html
Not sure which artchitecture the vllm models fall under?
It needs to be supported because Llama 3 is a fine-tuned version. LlamaForCausalLM
Thanks! what about the old SQL coder models?
sql-coder-7b-2, sql-coder-70b-alpha, sql-coder-34b-alpha
You can find information in the Model Details section. CodeLlama-7B for sqlcoder-7b-2 which also falls into the llama type --> LlamaForCausalLM
Yes it is supported by vllm, and @omeryentur rightly pointed out that these belong to the Llama family of models. In addition, our public evaluation framework + dataset allow you to use VLLM as the runner (see https://github.com/defog-ai/sql-eval?tab=readme-ov-file#vllm)