qualcomm

/

Llama-v2-7B-Chat

like 17

We are aware of this issue. We currently can't distribute the quantized weights so we are working on a way to produce these weights automatically. Stay tuned! This is the highest priority for us.

Do you have any guidance on converting the llama model to the qnn model @srikris

@eoe @julien-c We are working on publishing recipes for you to easily quantize this model and produce the weights. Should have something soon.

@srikris Hi Krishna, any progress on the matter, how can we convert the model and use it?

Almost there @ShaneRush . We have figured it out. Now just testing and polishing.

exciting stuff!

Any news?

Targeting the next release. (We release roughly every 2 weeks)

Llama2 with pre-computed quantization encodings is released.
Now you can use AI Hub Llama2 model to export models to QNN context binary and verify with demo.py on-device.