🦙 Llama-3.2-1B-ViLA

ViLA (Virtual Library Assistant) is a fine-tuned version of Meta's Llama-3.2-1B-Instruct model, optimized to answer library-related questions accurately and concisely. It has been instruction-tuned on a curated dataset of library FAQs, making it a lightweight yet capable assistant for library information services.

---

📋 Model Details

Attribute	Details
Base Model	meta-llama/Llama-3.2-1B-Instruct
Model Size	1 Billion parameters
Context Length	128K tokens
Fine-tuning Method	LoRA (Low-Rank Adaptation)
Training Dataset	hungryfoxz/LibraryFAQ1000
Framework	Hugging Face Transformers + PEFT + TRL
License	Llama 3.2 Community License (inherited from base model)

---

🎯 Intended Uses

This model is designed for:

• Library Information Services – Answering questions about memberships, policies, services, and general library operations
• On-Device Chatbots – Lightweight enough to run on edge devices and mobile platforms
• Educational Demonstrations – Showcasing fine-tuning of small language models for domain-specific tasks

The model is not intended for:

• General-purpose conversation beyond library-related topics
• Multimodal tasks (text-only model)
• High-stakes decision-making

---

🏋️ Training Details

Dataset

The model was fine-tuned on the hungryfoxz/LibraryFAQ1000 dataset, which contains over 1,000 question-answer pairs covering common library inquiries.

Training Configuration

Parameter	Value
LoRA Rank (r)	16
LoRA Alpha (lora_alpha)	32
LoRA Dropout	0.05
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Epochs	3
Batch Size	1 (with gradient accumulation over 8 steps)
Learning Rate	2e-4
Scheduler	Cosine with 3% warmup
Precision	FP16

Training Time

~10 minutes on Apple Silicon (M2) hardware.

---

🚀 Quick Start

---

Installation

If possible create a python environment for installing the packages. After activating the environment, create a file within the working directory, paste the code from the two section into that file, save it and run.

pip install torch transformers accelerate peft trl datasets huggingface_hub

---

Loading the Model

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
from huggingface_hub import login

# login("your_hf_token_here")
# Create a token from hf hub and paste here, its ***free***. Uncomment the above line after pasting the token.

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype=torch.float16
)

# Load the ViLA adapter
model = PeftModel.from_pretrained(base_model, "hungryfoxz/Llama-3.2-1B-ViLA")
model.eval()

---

Running Inference

def ask_question(question: str) -> str:
    messages = [{"role": "user", "content": question}]
    
    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        return_dict=True
    )
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=200,
            temperature=0.7,
            do_sample=True
        )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example
print(ask_question("Who are you?"))

Example Output

**User**: Who are you?
**Assistant**: I am a Virtual Library Assistant.

---

📊 Evaluation

The model was evaluated on the test split of the LibraryFAQ1000 dataset. Training metrics:

Step	Training Loss	Validation Loss	Mean Token Accuracy
100	1.645	1.505	66.8%
200	1.111	1.340	69.0%
300	0.791	1.395	68.8%

---

🖥️ Hardware Requirements

Precision	Disk Size	RAM (Inference)
BF16	~2 GB	~2-3 GB
4-bit (Q4)	~800 MB	~1 GB

The model runs comfortably on:

• Apple Silicon (M1/M2/M3) via MPS/ Intel i5 11th Gen / Ryzen 5 5000 series
• Mid-range laptops
• Edge devices (Raspberry Pi 5)
• macOs/ Windows / Linux or Unix

---

⚠️ Limitations

Domain-Specific – Optimized for library FAQs; performance on out-of-domain questions may degrade Small Model – As a 1B parameter model, it may lack the depth of larger models for complex reasoning Hallucination – Like all LLMs, may occasionally generate incorrect or fabricated information

---

🤝 Acknowledgements

• Meta for releasing the Llama 3.2 family of models
• Hugging Face for the Transformers, PEFT, and TRL libraries
• Dataset Contributors for the LibraryFAQ1000 dataset

---

Happy building! 📖✨

• Facing any issues‼️ while running the code, feel free to send a 📸screenshot to 'kaustavkhanikar99@gmail.com'. I will try to get back to you as soon as possible ✌️.