Error when running on M1

#8
by dpang - opened

Had the model worker running and on the first query, it died with the following message:

error: input types 'tensor<1x577x1xf16>' and 'tensor<1xf32>' are not broadcast compatible

My model worker command line is:

python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path ../llava-v1.5-13b --device mps

Sign up or log in to comment