Duranta19/llama32-3b-meta-ads-debugger-lora

Llama-3.2-3B Meta Ads Campaign Debugger (LoRA)

A LoRA adapter fine-tuned on top of Llama-3.2-3B-Instruct for Meta Ads campaign debugging — diagnosing campaign issues, explaining performance problems, and suggesting fixes for ad configurations.

This repository contains only the LoRA adapter (~95 MB), not the full base model. The base model weights are loaded separately at inference time.

Quick Start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "unsloth/Llama-3.2-3B-Instruct"
ADAPTER_ID = "Duranta19/llama32-3b-meta-ads-debugger-lora"

# Load tokenizer and model (base + LoRA adapter)
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
base = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    dtype=torch.float16,   # use torch.bfloat16 on Ampere+ GPUs (A100, RTX 30xx+)
    device_map="auto",
)
model = PeftModel.from_pretrained(base, ADAPTER_ID)
model.eval()

# Prepare input
system_prompt = "You are an expert Meta Ads Campaign Debugger. Return only valid JSON."
user_input = """Analyze the following Meta Ads performance data.
Campaign: TEST_01
Impressions: 50000
Clicks: 250
CTR: 0.5%
CPC: 4.50
Spend: 1125
Reach: 48000
Frequency: 1.04
CPM: 22.50"""

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_input},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        inputs,
        max_new_tokens=256,
        temperature=0.1,
        do_sample=True,
        pad_token_id=tokenizer.pad_token_id,
    )

# Decode only the newly generated tokens (skip the prompt)
response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

Example output:

{
  "campaign": "TEST_01",
  "diagnosis": "Low CTR (0.5%) with high CPC ($4.50) indicates weak ad relevance...",
  "issues": ["low_ctr", "high_cpc"],
  "recommendations": ["Test new creatives", "Refine audience targeting"]
}

Optional: merge the adapter for standalone deployment

merged = model.merge_and_unload()
merged.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")

Model Details

Base model	Llama-3.2-3B-Instruct (3.2B params)
Method	Supervised fine-tuning (SFT) with LoRA
Trainable params	24.3M (0.75% of total)
Precision	fp16
Frameworks	TRL SFTTrainer + PEFT
Hardware	2x NVIDIA T4 (Kaggle)

LoRA configuration

Hyperparameter	Value
Rank (r)	16
Alpha	32
Dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Training configuration

Hyperparameter	Value
Dataset size	10,000 samples (7,000 train / 2,000 val / 1,000 test)
Max sequence length	2048
Effective batch size	8
Learning rate	2e-4 (cosine schedule, 20 warmup steps)
Weight decay	0.01
Packing	Enabled
Best model selection	Lowest eval loss

Intended Use

Built as a demo for assisting with Meta (Facebook/Instagram) ads campaign troubleshooting: interpreting metrics (CPM, CTR, ROAS, frequency), spotting misconfigurations, and recommending next steps.

Limitations

• Trained on a 10K sample subset for demonstration purposes; coverage of rare campaign scenarios is limited.
• Advertising platforms change frequently — the model's knowledge reflects its training data and may not match current Meta Ads features or policies.
• Outputs are suggestions, not guarantees; validate recommendations before applying them to live ad spend.
• Inherits the biases and limitations of the Llama-3.2-3B-Instruct base model.

License

The base model is subject to the Llama 3.2 Community License. Use of this adapter requires compliance with that license.