ApyHTML19/Gemma-2b-It_EU-AI-Reglementation

🧠 Gemma-2B-IT LoRA Fine-Tuned

This model is a fine-tuned version of google/gemma-2b-it using LoRA (Low-Rank Adaptation) on a custom instruction dataset.

Model Details

• Base model: google/gemma-2b-it
• Fine-tuning method: LoRA (PEFT)
• Quantization: 4-bit (NF4) during training
• Framework: HuggingFace Transformers + TRL + PEFT
• Hardware: NVIDIA GPU /AWS)

Training Details

Parameter	Value
LoRA rank (r)	8
LoRA alpha	16
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj
Learning rate	2e-4
Epochs	3
Batch size	1
Gradient accumulation	16 steps
Optimizer	paged_adamw_8bit
Max sequence length	256

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model = "google/gemma-2b-it"
adapter = "your-username/your-repo-name"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter)

prompt = "<start_of_turn>user\nYour question here<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Dataset Format

The model was trained on instruction/response pairs in the following format: https://huggingface.co/datasets/ApyHTML19/EU-AI-Regulation-GDPR-2025