bosaj/eniad-llama3.1-8b-assistant-lora

🎓 ENIAD Assistant — LLaMA-3.1-8B LoRA Adapter

🤝 Official Multi-Author Engineering Project • ENIAD AI Lab (May 2025)

---

👥 The ENIAD AI Engineering Team

This model and its surrounding ecosystem were engineered as part of the Projet de Fin d'Année (PFA) at the National School of Artificial Intelligence and Digital (ENIAD), Mohammed First University, Oujda, Morocco:

AI Engineer	Official Role	Core Contributions	Profile Links
Abdellah ENNAJARI	Lead AI & MLOps Engineer	Microservice System Architecture, CI/CD Pipeline Automation, Multi-stage Docker Containerization	@abdennajari • GitHub @ennajari
Ahmed OUKACHA	AI Systems & Fine-Tuning Specialist	Custom Fine-Tuned LLaMA-3 8B Academic Checkpoint, Model Weights Optimization & Modal Platform API	@ahmed-ouka
Oussama EL HADJI	Full-Stack AI UI & SMA Multi-Agent Engineer	React 18 + Vite Conversational UI, Streaming Web Inference, SMA Multi-Agent Web Intelligence Service	HF @bosaj • GitHub @Bosaj • Portfolio
Abdelilah OURTI	Vector DB & RAG Pipeline Engineer	LanceDB / Qdrant Vector Store Indexing, Academic Document Embedding Pipelines, Fast Search Backend	@abdelilahou

---

🏛️ The Complete ENIAD Chatbot Model Family (LLaMA & Qwen)

The ENIAD Chatbot ecosystem consists of collaborative LLaMA & Qwen checkpoints, quantization profiles, and LoRA adapters engineered for our institutional assistant:

Model & Checkpoint	Base Architecture	Precision / Format	Target Environment & Use-Case	Verified Repository Link
LLaMA-3.1-8B Assistant LoRA	meta-llama/Llama-3.1-8B	16-Bit PEFT LoRA (Rank 16, Alpha 32)	Primary institutional conversational assistant (Bilingual FR/EN)	bosaj/eniad-llama3.1-8b-assistant-lora
LLaMA-3-8B Merged 32-Bit	meta-llama/Meta-Llama-3-8B	Full Float32 Merged Weights	Standalone backend inference server without runtime adapter loading	ahmed-ouka/llama3-8b-eniad-merged-32bit
LLaMA-3.1-8B Team Milestone	meta-llama/Llama-3.1-8B	PEFT LoRA Sharded Safetensors	Original PFA milestone model checkpoint (May 2025 team release)	ahmed-ouka/my-llama3.1-8B-with-lora-Eniad-Assistant
Eniad LLaMA 8-Bit Quantized	meta-llama/Meta-Llama-3-8B	8-Bit bitsandbytes NF4/INT8	High-efficiency local inference on edge GPUs (< 6GB VRAM)	ahmed-ouka/Eniad-model-llama-Assistant
LLaMA-Factory 3.1 LoRA Adapter	meta-llama/Llama-3.1-8B	Modular PEFT Adapter (~50MB)	Exported modular weights from the LLaMA-Factory training pipeline	ahmed-ouka/llama-lora-adapter-eniad
Qwen-2.5-1.5B ENIAD LoRA	Qwen/Qwen2.5-1.5B-Instruct	16-Bit PEFT LoRA (Compact)	Ultra-fast lightweight assistant for low-latency & edge devices (< 2GB VRAM)	ahmed-ouka/lora-qwen-eniad

---

📌 Model Overview

• Base Architecture: meta-llama/Meta-Llama-3.1-8B-Instruct
• Original Model Weights Checkpoint (May 2025): ahmed-ouka/my-llama3.1-8B-with-lora-Eniad-Assistant
• Live Interactive Web App: bosaj/chat_model
• Bilingual Instruction Dataset: bosaj/eniad-assistant-instruct-dataset
• Primary Institution: École Nationale d'Intelligence Artificielle et du Digital (ENIAD), UMP, Morocco.

---

🚀 How to Use (Inference Code)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
# Uses the original May 2025 team checkpoint
adapter_id = "ahmed-ouka/my-llama3.1-8B-with-lora-Eniad-Assistant"

bnb_config = BitsAndBytesConfig(
    load_in_8bit=True,
    torch_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

messages = [
    {"role": "system", "content": "You are the official ENIAD AI Assistant. Provide structured, accurate academic guidance."},
    {"role": "user", "content": "Quelles sont les spécialités proposées à l'ENIAD en cycle ingénieur ?"}
]

input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9)
response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

📊 Training Hyperparameters

Hyperparameter	Value	Description
LoRA Rank ($r$)	16	Rank dimension for low-rank adapter matrices
LoRA Alpha ($lpha$)	32	Scaling factor for adapter updates
LoRA Dropout	0.05	Regularization dropout rate
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj	Full attention & MLP projections
Optimizer	paged_adamw_8bit	Memory-efficient 8-bit optimizer
Learning Rate	2e-4	Cosine decay with warmup

---

📈 Evaluation & Benchmark Results

Benchmark Metric	Score	Baseline LLaMA-3.1-8B	Improvement
ROUGE-1	52.4	38.2	+14.2
ROUGE-2	28.1	16.5	+11.6
ROUGE-L	48.6	33.7	+14.9
Institutional Accuracy	94.8%	61.2%	+33.6%

---

📖 Citation

@misc{ennajari_ouka_elhadji_ourti_2025,
  author = {Ennajari, Abdellah and Oukacha, Ahmed and El Hadji, Oussama and Ourti, Abdelilah},
  title = {ENIAD Assistant: Parameter-Efficient Fine-Tuning of LLaMA-3.1-8B for Academic Mentorship and Institutional Intelligence},
  year = {2025},
  month = {May},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bosaj/eniad-llama3.1-8b-assistant-lora}}
}