# Bringing SOTA quantization to mobile LLM deployment: A practical Executorch integration guide

Article: https://blacksamorez.substack.com/p/aqlm-executorch-android

## Usage

 - Download and install the `.apk` file on your Android phone. (`llama3-aqlm.apk` for ~1.1 tok/s at low power consumption. `llama3-aqlm-4cores.apk` for ~2.7 tok/s at high loads)
 - Download the `.pte` and `.model` files and put them into the `/data/local/tmp/llama` folder on your Android phone.
 - Running the app you will see the option to load the `.pte` and `.model` files. After loading them, you'll be able to chat with the model.

## Requirements

This app was tested on `Samsung S24 Ultra` running `Android 14`.

## Limitations

 - Although the app looks like chat, generation requests are independent.
 - Llama-3 chat template is hard-coded into the app.