Llama 3.2 for Snapdragon NPU
Llama 3.2 models (1B and 3B) compiled for Qualcomm Snapdragon's NPU. These run natively on-device using the Hexagon processor - no cloud, no GPU needed.
What's in here
This repo has the compiled model binaries for running Llama 3.2 on Snapdragon devices:
- llm/ - Llama 3.2 3B Instruct (W4A16 - 4-bit weights, 16-bit activations)
- llama32_1b/ - Llama 3.2 1B Instruct (W4 - 4-bit weights)
Both models are split into parts due to file size. The 1B model includes separate token generation parts.
Supported devices
These work on Snapdragon chipsets with an NPU:
- Snapdragon 8 Elite
- Snapdragon 8 Gen 3
- Snapdragon 8 Gen 2
Basically if you have a flagship Android phone from the last ~2 years, you're good.
How to use
These models are meant to be used with Qualcomm's QNN SDK and the Genie runtime. You can't just drop them into any LLM app - they're compiled specifically for Snapdragon's architecture.
If you want a working Android app that uses these, check out the ChatApp implementation (uses the same model format).
Performance
On a Samsung Galaxy S25 Ultra (Snapdragon 8 Elite):
- 3B model: ~20-30 tokens/second
- 1B model: Should hit 50-150 tk/s (lighter model, faster inference)
All running 100% on-device using the NPU. No internet required after download.
Technical details
- Quantization:
- 3B: W4A16 (4-bit weights, 16-bit activations)
- 1B: W4 (4-bit weights)
- Context length: 4096 tokens
- Backend: Qualcomm HTP (Hexagon Tensor Processor)
- Compiled with: Qualcomm AI Hub
The models use runtime path placeholders in the config that get replaced by the app at load time. This makes them portable across different Android storage configurations.
Why this exists
Most on-device LLMs either run on the GPU (battery killer) or use generic CPU inference (slow). Snapdragon's NPU is purpose-built for this stuff and it shows - you get decent inference speed without destroying your battery.
Plus it's genuinely cool to have a 3 billion parameter model running entirely on your phone.
License
These are Meta's Llama 3.2 models, so they're under Meta's Llama license. Check Meta's official license terms before using in production.
Models were compiled using Qualcomm's tools but the underlying architecture is Meta's IP.
Download URLs
Direct links if you want to grab individual files:
https://huggingface.co/Eternalgrey/llama-snapdragon-models/res olve/main/llm/[filename] https://huggingface.co/Eternalgrey/llama-snapdragon-models/res olve/main/llama32_1b/[filename]
Replace [filename] with the actual .bin file name.