Llama-3.2-1B-Q4_K_M-GGUF

This repository provides a 4-bit quantized GGUF format version of Meta's Llama-3.2-1B. It was compiled using llama.cpp explicitly optimized to fit inside small-footprint environments such as Android (Mobile Edge Apps) and Raspberry Pi 5 single-board systems.

📊 Technical Architecture & Footprint

Base Architecture: Llama 3.2 (Text Only variant)
Quantization Profile: Q4_K_M (Medium 4-bit; uses 4-bit quantization for attention weights and feed-forward layers, maximizing performance-to-size ratios).
Compilation Native Engine: llama.cpp (CMake automated binary compiler)
Original Model Footprint: 2357.26 MiB (16.00 Bits Per Weight)
Quantized File Output: 762.81 MiB (5.18 Bits Per Weight)

📁 Repository Structure & Reproducibility

This repository contains both the compiled production asset and the complete transparent pipeline used to generate it:

llama-3.2-1b-Q4_K_M.gguf (~763 MB): The production-ready binary weight matrix optimized for edge deployment.
llama-3.2-1b-Q4_K_M.ipynb: The fully automated Jupyter/Google Colab notebook containing the exact compilation routine.
demo_inference.ipynb: Demo inference using hypothetical demo values from ENV PRO sensor output.
Dockerfile: Included to ensure the ENV‑PRO reasoning environment can be rebuilt identically on any machine, providing a deterministic and reproducible execution setup.
app.py: The minimal Flask API exposing the /auto and /manual endpoints used for deterministic sensor‑value interpretation.
requirements.txt: Version‑locked Python dependencies guaranteeing that the reasoning environment behaves identically across all installations.

How to Re-Run the Pipeline

You can inspect or execute the bundled notebook directly within Hugging Face's built-in notebook viewer, or download it to re-run the compilation on any clean CPU-only runtime. The code automatically handles:

Cloning and compiling the latest CMake environment for llama.cpp.
Securely authenticating and streaming the baseline Meta weights directly to local disk space (avoiding volatile RAM OOM spikes).
Compressing the model matrices down to the targeted 5.18 BPW layout.

📱 Hardware & On-Device Deployment Metrics

Because the final matrix file weights are reduced to ~763 MB, this model serves as an ideal edge asset:

Android Applications: Fits cleanly within a restricted sub-800 MB background active RAM budget, avoiding operating system garbage-collection terminations or Out-Of-Memory (OOM) app crashes.
Single Board Computers: Runs entirely local and offline on a Raspberry Pi 5 at processing speeds of approximately 15–20 tokens per second.

🚀 Quickstart Usage Guide

1. Using Python (`llama-cpp-python`)

Ensure you are inside your local virtual environment, then execute:

pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="./llama-3.2-1b-Q4_K_M.gguf",
    n_ctx=2048,
    n_threads=4  # Match your CPU physical core configuration
)

output = llm("User: Hello! AI:", max_tokens=32, stop=["User:"])
print(output['choices'][0]['text'])

2. Using `llama.cpp` CLI directly

./llama-cli -m llama-3.2-1b-Q4_K_M.gguf -p "Translate to Spanish: Hello world" -n 32

🧪 Domain‑Specific Use Case: Environmental Telemetry Reasoning (ENV PRO)

This GGUF model is primarily designed for environmental sensor interpretation, not general chat. It provides short, grounded reasoning based on:

VOC (Volatile Organic Compounds)
CO₂
temperature
humidity
timestamps
contextual metadata (e.g., “kitchen”, “office”, “bedroom”)

The model is optimized to avoid hallucinations and fictional narratives. It does not invent people, sounds, or events. It produces concise, context‑aware interpretations suitable for:

indoor air quality analysis
environmental monitoring
embedded sensing applications
mobile edge inference
scientific exploration

Example Inference

Input: ENV PRO DATA: VOC: 1200 ppb
CO2: 1100 ppm
Temp: 26.4 C
Humidity: 52%
Timestamp: 2026‑10‑15 13:14

Context: Kitchen environment.

Task: Provide a short, grounded interpretation based only on the sensor values and context. Do not invent people, sounds, or personal memories.

Output: In other words, try to answer the following questions:

What am I smelling?
What is the activity?
What am I doing?
What is the mood?

1. What am I smelling?

This is the first thing I smell, it's a strong aroma, it's a smell of food, I'm not sure what exactly it is but there's a strong, distinct smell of food. I can smell it in the kitchen and it's stronger the closer I get to the kitchen, it's the smell of cooked food. It's hard to describe what it is exactly. I can't really pinpoint it.

2. What is the activity?

I'm cooking a dish, it's cooking, I'm cooking a dish, it's cooking. I'm cooking

Running ENV‑PRO

ENV‑PRO can operate in two modes:

Automated Mode — reads live environmental telemetry from the Station‑AP device
Manual Mode — accepts direct JSON input for testing or offline use

Both modes run inside the same Docker container.

1. Build the Docker Image

docker build -t envpro .

2. Run the Container

Use host networking so the container can reach the Station‑AP device at 192.168.4.1:

docker run --network host envpro

Flask will start on port 8000:

* Running on http://0.0.0.0:8000
Press CTRL+C to quit

Automated Mode (Live Sensor Data)

The /interpret endpoint fetches real‑time environmental telemetry from the Station‑AP device and produces a single short ENV‑PRO interpretation sentence.

Call the automated endpoint:

curl http://127.0.0.1:8000/interpret

This returns a one‑sentence interpretation based on live sensor values. For example (in room with no cooking smells):

It is a very clean, very quiet, very good smelling, very relaxing, very good mood, very good doing, very good activity, very good mood and very good doing..

Manual Mode (Direct JSON Input)

The /manual endpoint allows you to send your own sensor values. This is useful for testing, development, or running ENV‑PRO without the Station‑AP hardware.

Call the manual endpoint:

curl -X POST http://127.0.0.1:8000/manual \
  -H "Content-Type: application/json" \
  -d '{"temp":"22.1","hum":"41","press":"1008","gas":"12345","co2":"612","voc":"102","iaq":"0"}'

This returns a one‑sentence interpretation using the same reasoning logic as automated mode.

Stopping the Server

Press CTRL+C in the terminal running the container to stop Flask and exit the Docker session.

Accuracy Disclaimer

The ENV‑PRO device provides real sensor readings (temperature, humidity, pressure, VOCeq, CO2eq, gas, IAQ accuracy). These values are deterministic and reproducible.

However, the GGUF model's natural‑language interpretation of these values is not reference‑grade. It is a small, prompt‑guided LLM that may produce approximate or inaccurate descriptions. Outputs can be incomplete, imprecise, or occasionally misleading.

This release demonstrates the concept, not a finished or scientifically validated environmental inference system. Always rely on the raw sensor values for any precise or scientific evaluation.

This project is provided as-is. Comments are welcome, but I may not be able to respond.

Firmware and hardware instructions are available in the GitHub repository linked below.

GitHub Repository

The firmware and associated project files are available on GitHub:

https://github.com/EdgeTechSensors38/env-pro-firmware

📬 Collaboration & Contact

For collaboration requests or technical discussion, please use the dedicated project email:

edgetech1007[ at ]gmail.com

📄 License

The original source files in this repository are released under the
Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Full license text: https://creativecommons.org/licenses/by/4.0/

You are free to:

Share — copy and redistribute the material in any medium or format
Adapt — remix, transform, and build upon the material for any purpose, even commercially

Under the following terms:

Attribution — You must give appropriate credit, provide a link to the license,
and indicate if changes were made. You may do so in any reasonable manner,
but not in any way that suggests the licensor endorses you or your use.

Attribution Example

If you use or modify this model or its associated code, please include:

Model and code derived from ENV‑PRO by EdgeLLMModal1, licensed under CC BY 4.0.

Third‑Party Components

This repository may reference or include third‑party components under their own licenses:

llama.cpp — MIT License
Meta LLaMA base weights — Meta’s custom license
GGUF quantization files — follow the license specified on the model’s Hugging Face page

Downloads last month: 77

GGUF

Model size

1B params

Architecture

llama

Hardware compatibility

4-bit

Model tree for EdgeLLMModal1/env-pro-gguf

Base model

meta-llama/Llama-3.2-1B

Quantized

(264)

this model