TsipiDev/Thesis_Models_Dimitris_Vatousis_220007

Thesis Models: Energy-Efficient LLM Inference on the Raspberry Pi 400

This repository hosts the nine GGUF model files used in the experiments of the undergraduate dissertation "Research and Development of Energy-Efficient Inference Methods for Generative AI (LLMs) on Low-Power Embedded Systems" (D. N. Vatousis, Department of Informatics, Democritus University of Thrace, Kavala, 2026).

Experiment code: https://github.com/TsipiDev/Thesis_Code_Dimitris_Vatousis_220007

Files

File	Base model	Variant
tinyllama_fp16.gguf	TinyLlama/TinyLlama-1.1B-Chat-v1.0	F16
tinyllama_q8.gguf	TinyLlama/TinyLlama-1.1B-Chat-v1.0	Q8_0 (INT8)
tinyllama_pruned.gguf	TinyLlama/TinyLlama-1.1B-Chat-v1.0	F16, 30% unstructured L1 pruning
llama3.2_fp16.gguf	meta-llama/Llama-3.2-1B	F16
llama3.2_q8.gguf	meta-llama/Llama-3.2-1B	Q8_0 (INT8)
llama3.2_pruned.gguf	meta-llama/Llama-3.2-1B	F16, 30% unstructured L1 pruning
qwen2_fp16.gguf	Qwen/Qwen2-0.5B	F16
qwen2_q8.gguf	Qwen/Qwen2-0.5B	Q8_0 (INT8)
qwen2_pruned.gguf	Qwen/Qwen2-0.5B	F16, 30% unstructured L1 pruning

Pruning was applied with torch.nn.utils.prune.l1_unstructured (amount 0.3) to every torch.nn.Linear layer, made permanent with prune.remove, without retraining. Conversion used llama.cpp convert_hf_to_gguf.py (--outtype f16) and quantization used llama-quantize (Q8_0).

These are research artifacts for reproducing the thesis measurements. The pruned variants are intentionally degraded (the pruned Llama 3.2 in particular produces repetitive text).

Download

pip install -U huggingface_hub
hf download TsipiDev/Thesis_Models_Dimitris_Vatousis_220007 --local-dir models/gguf

Licenses

Each file inherits the license of its base model:

• TinyLlama and Qwen2 derivatives: Apache License 2.0.
• Llama 3.2 derivatives: Llama 3.2 Community License Agreement (https://huggingface.co/meta-llama/Llama-3.2-1B/blob/main/LICENSE.txt), subject to Meta's Acceptable Use Policy. Built with Llama.