Skill: ivangdavila/fine-tuning (version 1.0.0)

--- name: Fine-Tuning slug: fine-tuning description: Fine-tune LLMs with data preparation, provider selection, cost estimation, evaluation, and compliance checks. --- ## When to Use User wants to fine-tune a language model, evaluate if fine-tuning is worth it, or debug training issues. ## Quick Reference | Topic | File | |-------|------| | Provider comparison & pricing | `providers.md` | | Data preparation & validation | `data-prep.md` | | Training configuration | `training.md` | | Evaluation & debugging | `evaluation.md` | | Cost estimation & ROI | `costs.md` | | Compliance & security | `compliance.md` | ## Core Capabilities 1. **Decide fit** — Analyze if fine-tuning beats prompting for the use case 2. **Prepare data** — Convert raw data to JSONL, deduplicate, validate format 3. **Select provider** — Compare OpenAI, Anthropic (Bedrock), Google, open source based on constraints 4. **Estimate costs** — Calculate training cost, inference savings, break-even point 5. **Configure training** — Set hyperparameters (learning rate, epochs, LoRA rank) 6. **Run evaluation** — Compare fine-tuned vs base model on task-specific metrics 7. **Debug failures** — Diagnose loss curves, overfitting, catastrophic forgetting 8. **Handle compliance** — Scan for PII, configure on-premise training, generate audit logs ## Decision Checklist Before recommending fine-tuning, ask: - [ ] What's the failure mode with prompting? (format, style, knowledge, cost) - [ ] How many training examples available? (minimum 50-100) - [ ] Expected inference volume? (affects ROI calculation) - [ ] Privacy constraints? (determines provider options) - [ ] Budget for training + ongoing inference? ## Fine-Tune vs Prompt Decision | Signal | Recommendation | |--------|----------------| | Format/style inconsistency | Fine-tune ✓ | | Missing domain knowledge | RAG first, then fine-tune if needed | | High inference volume (>100K/mo) | Fine-tune for cost savings | | Requirements change frequently | Stick with prompting | | <50 quality examples | Prompting + few-shot | ## Critical Rules - **Data quality > quantity** — 100 great examples beat 1000 noisy ones - **LoRA first** — Never jump to full fine-tuning; LoRA is 10-100x cheaper - **Hold out eval set** — Always 80/10/10 split; never peek at test data - **Same precision** — Train and serve at identical precision (4-bit, 16-bit) - **Baseline first** — Run eval on base model before training to measure actual improvement - **Expect iteration** — First attempt rarely optimal; plan for 2-3 cycles ## Common Pitfalls | Mistake | Fix | |---------|-----| | Training on inconsistent data | Manual review of 100+ samples before training | | Learning rate too high | Start with 2e-4 for SFT, 5e-6 for RLHF | | Expecting new knowledge | Fine-tuning adjusts behavior, not knowledge — use RAG | | No baseline comparison | Always test base model on same eval set | | Ignoring forgetting | Mix 20% general data to preserve capabilities |

path: compliance.md
content: # Compliance & Security

## PII Detection

### Before Training: Scan Everything

```python
import re

PII_PATTERNS = {
    "email": r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
    "phone": r'\b(\+?1?[-.\s]?)?\(?[0-9]{3}\)?[-.\s]?[0-9]{3}[-.\s]?[0-9]{4}\b',
    "ssn": r'\b\d{3}-\d{2}-\d{4}\b',
    "ip_address": r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b',
    "credit_card": r'\b(?:\d{4}[-\s]?){3}\d{4}\b',
}

def scan_for_pii(text):
    findings = []
    for pii_type, pattern in PII_PATTERNS.items():
        matches = re.finditer(pattern, text, re.IGNORECASE)
        for match in matches:
            findings.append({
                "type": pii_type,
                "value": match.group(),
                "position": match.span()
            })
    return findings

def audit_dataset(path):
    report = {"total_examples": 0, "pii_found": []}
    
    with open(path) as f:
        for i, line in enumerate(f, 1):
            obj = json.loads(line)
            report["total_examples"] += 1
            
            for msg in obj.get("messages", []):
                findings = scan_for_pii(msg.get("content", ""))
                if findings:
                    report["pii_found"].append({
                        "line": i,
                        "findings": findings
                    })
    
    return report
```

### PII Remediation Options

| Option | Pros | Cons |
|--------|------|------|
| Remove examples | Simple | Data loss |
| Redact with [PII] | Preserves structure | May confuse model |
| Pseudonymize | Preserves patterns | Complex to implement |
| Synthetic replacement | Best quality | Expensive |

## GDPR Compliance

### Article 6: Lawful Basis

Before training, document:
- [ ] **Consent** — Users agreed to AI training
- [ ] **Legitimate interest** — Business need documented
- [ ] **Contract** — Training necessary for service
- [ ] **Legal obligation** — Required by law

### Article 17: Right to Erasure

Requirements for compliant fine-tuning:
- Track which data points went into training
- Plan for re-training if erasure requested
- Document data retention periods

### Records of Processing (ROPA)

Document for each fine-tuning activity:
- Purpose of processing
- Categories of data subjects
- Data retention period
- Third-party processors (OpenAI, cloud GPUs)
- Technical safeguards

## On-Premise Training

### When Required
- Sensitive data cannot leave environment
- Regulated industry (healthcare, finance)
- Government contracts
- Internal policy restrictions

### Setup Options

| Tool | Requirements | Best For |
|------|--------------|----------|
| Unsloth | 24GB+ VRAM | Fast, easy setup |
| Axolotl | 48GB+ VRAM | Complex configs |
| vLLM + LoRA | Variable | Serving focus |
| MLX (Mac) | Apple Silicon | M1/M2/M3 local |

### Air-Gapped Training

```bash
# Pre-download everything
pip download torch transformers unsloth -d ./packages/
huggingface-cli download meta-llama/Llama-3.1-8B --local-dir ./models/

# Transfer to air-gapped system
# Install from local
pip install --no-index --find-links=./packages/ torch transformers unsloth
```

## Audit Logging

### What to Log

| Event | Data to Capture |
|-------|-----------------|
| Training start | Dataset hash, model, hyperparams, user |
| Training complete | Model hash, metrics, duration |
| Data access | Who, when, what dataset |
| Model deployment | Version, endpoint, approver |

### Audit Log Schema

```python
audit_log = {
    "timestamp": "2024-01-15T10:30:00Z",
    "event_type": "training_started",
    "actor": "[REDACTED_SECRET]",
    "resources": {
        "dataset_hash": "sha256:abc123...",
        "base_model": "meta-llama/Llama-3.1-8B",
        "output_path": "/models/ft-v1/"
    },
    "parameters": {
        "epochs": 3,
        "learning_rate": 2e-4,
        "lora_rank": 16
    },
    "justification": "Customer support automation project"
}
```

## Data Handling Checklist

Before training:
- [ ] PII scan completed, no findings or remediated
- [ ] Lawful basis documented
- [ ] Data retention policy defined
- [ ] Third-party DPA signed (if using cloud)
- [ ] Audit logging enabled
- [ ] Deletion procedure documented

After training:
- [ ] Training data securely deleted or archived
- [ ] Model documented in inventory
- [ ] Access controls configured
- [ ] Monitoring enabled

## Memorization Risk Assessment

Fine-tuned models can memorize training data. Test for:

```python
def test_memorization(model, training_examples, sample_size=100):
    """Check if model regurgitates exact training data"""
    
    risky = []
    for ex in random.sample(training_examples, sample_size):
        # Give model the beginning of a training example
        prompt = ex["messages"][1]["content"][:50]  # First 50 chars of user msg
        
        completion = generate(model, prompt)
        
        # Check if completion matches training data
        expected = ex["messages"][2]["content"]
        if similarity(completion, expected) > 0.9:
            risky.append(ex)
    
    return len(risky) / sample_size  # Memorization rate
```

If memorization rate >5%, consider:
- More diverse training data
- Differential privacy during training
- Smaller model or fewer epochs


path: costs.md
content: # Cost Estimation & ROI

## Training Cost Calculation

### OpenAI Formula

```
Training Cost = (tokens_in_dataset × epochs × cost_per_M_tokens) / 1,000,000
```

### Example: 1M Token Dataset

| Model | Training Cost |
|-------|---------------|
| GPT-4o | $25 × 3 epochs = $75 |
| GPT-4o-mini | $3 × 3 epochs = $9 |
| GPT-4.1-nano | $1.50 × 3 epochs = $4.50 |

### Open Source (Cloud GPU)

| GPU | Cost/Hour | Typical Training |
|-----|-----------|------------------|
| A100 80GB | $1.50-2.00 | 2-8 hours |
| RTX 4090 | $0.40-0.70 | 4-12 hours |
| Free Colab | $0 | Limited to small models |

## Inference Cost Comparison

### Per-Request Cost

| Setup | Cost per 1K tokens |
|-------|-------------------|
| GPT-4o base | $2.50 (in) + $10 (out) |
| GPT-4o fine-tuned | $3.75 (in) + $15 (out) |
| GPT-4o-mini base | $0.15 (in) + $0.60 (out) |
| GPT-4o-mini fine-tuned | $0.30 (in) + $1.20 (out) |
| Self-hosted Llama 8B | ~$0.01 (GPU amortized) |

**Note:** Fine-tuned models on OpenAI cost MORE per token. ROI comes from shorter prompts or using smaller models effectively.

## Break-Even Analysis

### When Fine-Tuning Saves Money

Fine-tuning pays off when:
1. **Shorter prompts** — No need for long system prompts or examples
2. **Smaller model works** — Fine-tuned mini matches base large
3. **High volume** — Fixed training cost amortized over many requests

### Example Calculation

**Scenario:** 100K requests/month, 500 tokens average

**Option A: GPT-4o with long prompt (2000 token context)**
- Input: 100K × 2500 tokens × $2.50/1M = $625
- Output: 100K × 500 tokens × $10/1M = $500
- Monthly: $1,125

**Option B: Fine-tuned GPT-4o-mini (500 token context)**
- Training: $10 (one-time)
- Input: 100K × 1000 tokens × $0.30/1M = $30
- Output: 100K × 500 tokens × $1.20/1M = $60
- Monthly: $90

**Savings:** $1,035/month after first month

### Break-Even Calculator

```python
def calculate_breakeven(
    training_cost,
    base_cost_per_request,
    finetuned_cost_per_request,
    requests_per_month
):
    savings_per_request = base_cost_per_request - finetuned_cost_per_request
    
    if savings_per_request <= 0:
        return "Fine-tuning increases costs"
    
    requests_to_breakeven = training_cost / savings_per_request
    months_to_breakeven = requests_to_breakeven / requests_per_month
    
    return {
        "requests_to_breakeven": requests_to_breakeven,
        "months_to_breakeven": months_to_breakeven,
        "monthly_savings_after": savings_per_request * requests_per_month
    }
```

## Cost Optimization Strategies

### 1. Start Small
- Begin with GPT-4o-mini or Llama 8B
- Only upgrade if quality insufficient
- Test quality-per-dollar before committing

### 2. Use Batch API
- OpenAI Batch API: 50% discount
- Trade latency (24h window) for cost
- Good for non-real-time processing

### 3. Enable Data Sharing
- OpenAI offers 50% inference discount
- Trade privacy for cost
- Not suitable for sensitive data

### 4. Right-Size Your Model
| Volume | Recommendation |
|--------|----------------|
| <10K/month | Use base model with prompting |
| 10K-100K/month | Consider fine-tuning mini |
| 100K+/month | Fine-tuning ROI strong |

### 5. Self-Host at Scale
- Break-even vs API: typically 500K-1M requests/month
- Consider: GPU cost, maintenance, reliability
- Use QLoRA to minimize GPU requirements

## Hidden Costs to Consider

| Cost | Impact |
|------|--------|
| Data preparation | Engineering time |
| Iteration cycles | 2-3 training runs typical |
| Evaluation infrastructure | Compute for evals |
| Monitoring | Ongoing quality tracking |
| Retraining | When data distribution shifts |
| Expertise | Learning curve, debugging |


path: data-prep.md
content: # Data Preparation

## Format Requirements

### OpenAI Chat Format (Standard)
```jsonl
{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
```

### Multi-Turn Conversations
```jsonl
{"messages": [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "What's the weather?"},
  {"role": "assistant", "content": "I don't have access to weather data."},
  {"role": "user", "content": "Can you check?"},
  {"role": "assistant", "content": "No, I can't access external services."}
]}
```

## Minimum Dataset Sizes

| Use Case | Minimum | Recommended |
|----------|---------|-------------|
| Format/style adaptation | 50-100 | 200-500 |
| Classification | 200-500 | 1,000-2,000 |
| Complex reasoning | 500-1,000 | 2,000-5,000 |
| Domain expertise | 1,000+ | 5,000-10,000 |

**Quality > Quantity** — LIMA paper showed 1,000 high-quality examples beat 52,000 low-quality.

## Data Quality Checklist

Before training, verify:
- [ ] Consistent format across ALL examples
- [ ] No contradictory examples (same input, different outputs)
- [ ] Examples match production input distribution
- [ ] Edge cases included (10-20% of dataset)
- [ ] No duplicate or near-duplicate entries
- [ ] Proper train/val/test split (80/10/10)

## Validation Script

```python
import json
from collections import Counter

def validate_jsonl(path):
    errors = []
    examples = []
    
    with open(path) as f:
        for i, line in enumerate(f, 1):
            try:
                obj = json.loads(line)
                if "messages" not in obj:
                    errors.append(f"Line {i}: Missing 'messages' key")
                    continue
                    
                messages = obj["messages"]
                roles = [m["role"] for m in messages]
                
                # Check role sequence
                if not roles[-1] == "assistant":
                    errors.append(f"Line {i}: Must end with assistant")
                
                # Check for empty content
                for m in messages:
                    if not m.get("content", "").strip():
                        errors.append(f"Line {i}: Empty content")
                        
                examples.append(obj)
                
            except json.JSONDecodeError:
                errors.append(f"Line {i}: Invalid JSON")
    
    print(f"Total examples: {len(examples)}")
    print(f"Errors: {len(errors)}")
    for e in errors[:10]:
        print(f"  - {e}")
    
    return examples, errors
```

## Deduplication

```python
import hashlib

def dedupe_by_input(examples):
    seen = set()
    unique = []
    
    for ex in examples:
        # Hash user messages only
        user_msgs = [m["content"] for m in ex["messages"] if m["role"] == "user"]
        key = hashlib.md5(str(user_msgs).encode()).hexdigest()
        
        if key not in seen:
            seen.add(key)
            unique.append(ex)
    
    print(f"Removed {len(examples) - len(unique)} duplicates")
    return unique
```

## Synthetic Data Generation

When you need more examples:

```python
def augment_example(example, model="gpt-4o"):
    """Generate variations of an existing example"""
    prompt = f"""
    Given this training example, generate 3 variations that:
    - Keep the same intent and output format
    - Vary the phrasing and specifics
    - Maintain quality
    
    Original: {json.dumps(example)}
    """
    
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}]
    )
    
    return parse_variations(response.choices[0].message.content)
```

## Common Data Issues

| Issue | Detection | Fix |
|-------|-----------|-----|
| Inconsistent formats | Check output structure variance | Standardize template |
| Contradictions | Hash inputs, compare outputs | Manual review, remove |
| Distribution mismatch | Compare to production logs | Add production examples |
| Missing edge cases | Analyze failure modes | Targeted collection |
| Token length issues | Count tokens per example | Truncate or split |


path: evaluation.md
content: # Evaluation & Debugging

## Evaluation Strategy

### Always Compare Three Versions

| Version | Purpose |
|---------|---------|
| Base model + prompting | Baseline — what you're trying to beat |
| Base model + few-shot | Improved baseline |
| Fine-tuned model | Your candidate |

### Metrics by Task Type

| Task | Primary Metric | Secondary |
|------|----------------|-----------|
| Classification | F1, Accuracy | Precision, Recall |
| Generation (quality) | Human preference | LLM-as-judge |
| Generation (format) | Exact match rate | Format compliance % |
| Code | Pass@k | Execution success |
| QA | Exact match | ROUGE, semantic sim |
| Summarization | ROUGE-L | Human preference |

## Building Evaluation Sets

### Requirements
- **Size:** 100-500 examples minimum
- **Distribution:** Match expected production inputs
- **Edge cases:** 10-20% challenging examples
- **Labels:** Ground truth or reference outputs
- **Never seen during training** — Critical!

### Example Eval Structure

```python
eval_examples = [
    {
        "input": "User query or context",
        "expected": "Correct response",
        "category": "edge_case",  # for segmented analysis
        "difficulty": "hard"
    },
    # ...
]
```

## LLM-as-Judge Evaluation

Cost-effective for scale. Use stronger model to judge outputs.

```python
JUDGE_PROMPT = """
Rate this response on a scale of 1-5:
- 5: Perfect, no issues
- 4: Good, minor issues
- 3: Acceptable, some problems
- 2: Poor, significant issues
- 1: Unacceptable

Input: {input}
Response: {response}
Reference: {reference}

Score (1-5):
"""

def evaluate_with_judge(examples, model="gpt-4o"):
    scores = []
    for ex in examples:
        response = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": JUDGE_PROMPT.format(**ex)}]
        )
        score = extract_score(response.choices[0].message.content)
        scores.append(score)
    return sum(scores) / len(scores)
```

## A/B Comparison

Direct comparison between base and fine-tuned:

```python
COMPARE_PROMPT = """
Which response is better for this input?

Input: {input}

Response A:
{response_a}

Response B:
{response_b}

Winner (A or B or Tie):
"""

def ab_test(examples, base_model, finetuned_model):
    wins = {"base": 0, "finetuned": 0, "tie": 0}
    
    for ex in examples:
        base_response = generate(base_model, ex["input"])
        ft_response = generate(finetuned_model, ex["input"])
        
        # Randomize order to avoid position bias
        if random.random() > 0.5:
            a, b = base_response, ft_response
            mapping = {"A": "base", "B": "finetuned"}
        else:
            a, b = ft_response, base_response
            mapping = {"A": "finetuned", "B": "base"}
        
        winner = judge(ex["input"], a, b)
        if winner == "Tie":
            wins["tie"] += 1
        else:
            wins[mapping[winner]] += 1
    
    return wins
```

## Debugging Training Issues

### Loss Not Decreasing

| Cause | Diagnosis | Fix |
|-------|-----------|-----|
| LR too low | Loss barely moves | Increase LR 2-10x |
| Bad data | Random loss pattern | Audit data quality |
| Format mismatch | Model confused | Fix data format |

### Loss Decreasing Then Diverging

| Cause | Diagnosis | Fix |
|-------|-----------|-----|
| LR too high | Loss explodes mid-training | Reduce LR, add warmup |
| Gradient explosion | Grad norm >100 | Gradient clipping |

### Good Training, Bad Inference

| Cause | Diagnosis | Fix |
|-------|-----------|-----|
| Overfitting | Perfect train, bad eval | More data, regularization |
| Precision mismatch | Different behavior | Match train/serve precision |
| Distribution shift | Eval inputs differ | Align eval with training |

### Model Forgets General Capabilities

| Cause | Diagnosis | Fix |
|-------|-----------|-----|
| High LR | Base skills degraded | Lower LR |
| No diversity | Only task data | Mix 20% general data |
| Too many epochs | Overspecialized | Fewer epochs |

## Failure Mode Analysis

After evaluation, categorize failures:

```python
failure_categories = {
    "format": [],      # Wrong structure
    "factual": [],     # Incorrect information
    "incomplete": [],  # Missing required content
    "style": [],       # Wrong tone/voice
    "hallucination": [], # Made up content
    "other": []
}

# Analyze each failure, bucket it, identify patterns
# Use patterns to guide data augmentation for next training round
```


path: providers.md
content: # Provider Selection

## Quick Comparison

| Provider | Best For | Training Cost | Min Examples |
|----------|----------|---------------|--------------|
| OpenAI GPT-4o-mini | Cost-effective default | $3/M tokens | 50 |
| OpenAI GPT-4o | Max quality | $25/M tokens | 50 |
| Anthropic (Bedrock) | Claude-required projects | Per-hour | 1000+ |
| Llama + Unsloth | Privacy, budget | GPU hours | 100 |
| Mistral | Multilingual, EU | API or self-host | 100 |

## OpenAI Fine-Tuning

**Models available:**
- GPT-4o ($25/M training, $3.75/$15 inference)
- GPT-4o-mini ($3/M training, $0.30/$1.20 inference) — **default choice**
- GPT-4.1 / GPT-4.1-mini / GPT-4.1-nano (newest)
- o4-mini ($100/hour, reasoning fine-tuning)

**Strengths:**
- Best documentation and DX
- Results with as few as 50 examples
- Reinforcement Fine-Tuning (RFT) for reasoning
- 50% discount if you enable data sharing

**Process:**
```bash
# Validate format
openai api fine_tuning.jobs.validate_file -f training.jsonl

# Create job
openai api fine_tuning.jobs.create \
  -t "file-abc123" \
  -m "gpt-4o-mini-2024-07-18"
```

## Anthropic / Claude (via Bedrock)

**Status:** Only Claude 3 Haiku fine-tuning is GA (as of Nov 2024).

**Constraints:**
- Must use Amazon Bedrock (no direct API)
- Region: US West (Oregon) only
- No Claude 3.5 Sonnet or Opus fine-tuning

**Results:**
- +24.6% F1 improvement reported
- Fine-tuned Haiku outperforms base Sonnet by 9.9%

**When to use:** Only if Claude is mandatory AND Haiku quality is acceptable.

## Google / Gemini

**Available:** Gemini 1.5 Pro and Flash via Vertex AI.

**Notes:**
- Less community adoption
- Good for GCP-native teams
- Supports LoRA-style efficient tuning

## Open Source Stack

### Recommended Tooling

| Tool | Use Case | VRAM Required |
|------|----------|---------------|
| **Unsloth** | Fast LoRA/QLoRA, best for beginners | 3GB+ (QLoRA) |
| **Axolotl** | Complex configs, multi-method | 16GB+ |
| **PEFT** | HuggingFace ecosystem integration | Varies |
| **TorchTune** | Research, full control | 24GB+ |

### Base Models

| Model | Parameters | Strengths |
|-------|------------|-----------|
| Llama 3.1 8B | 8B | Best small model, fast training |
| Llama 3.1 70B | 70B | Near-GPT-4 quality |
| Mistral 7B | 7B | Multilingual, efficient |
| Qwen 2.5 | 7B-72B | Excellent multilingual |

### LoRA vs QLoRA

| Technique | Precision | VRAM Savings | Quality |
|-----------|-----------|--------------|---------|
| LoRA | 16-bit | ~50% | Higher |
| QLoRA | 4-bit | ~75% | Slightly lower |

**Recommendation:** Start with Unsloth Dynamic 4-bit QLoRA. Recovers most quality loss.

## Decision Matrix

| Constraint | Recommendation |
|------------|----------------|
| Default / best DX | OpenAI GPT-4o-mini |
| Budget <$50 | Unsloth + free Colab |
| Max quality | OpenAI GPT-4o with RFT |
| Privacy required | Llama + Axolotl locally |
| Claude required | Bedrock + Haiku |
| Multilingual | Qwen or Gemini |


path: training.md
content: # Training Configuration

## Hyperparameter Defaults

### For Supervised Fine-Tuning (SFT)

| Parameter | Default | Range | Notes |
|-----------|---------|-------|-------|
| Learning rate | 2e-4 | 1e-5 to 5e-4 | Lower for larger models |
| Epochs | 1-3 | 1-5 | More = overfitting risk |
| Batch size | 4-8 | 2-32 | Higher = more stable, more VRAM |
| Warmup ratio | 0.03 | 0.01-0.1 | 3% of total steps |
| Weight decay | 0.01 | 0-0.1 | Regularization |

### For LoRA/QLoRA

| Parameter | Default | Notes |
|-----------|---------|-------|
| Rank (r) | 16 | 8-64, higher = more capacity |
| Alpha | 16 | Usually equals rank |
| Dropout | 0 | 0-0.1, use if overfitting |
| Target modules | All attention | q, k, v, o, gate, up, down proj |

### For RLHF/DPO/GRPO

| Parameter | Default | Notes |
|-----------|---------|-------|
| Learning rate | 5e-6 | 10-50x lower than SFT |
| Beta (DPO) | 0.1 | Controls preference strength |
| Epochs | 1 | Usually single pass |

## Training with Unsloth (Recommended)

```python
from unsloth import FastLanguageModel

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B",
    max_seq_length=2048,
    load_in_4bit=True,  # QLoRA
)

# Apply LoRA
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", 
                    "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    use_gradient_checkpointing="unsloth",
)

# Train
from trl import SFTTrainer

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    args=TrainingArguments(
        per_device_train_batch_size=4,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=100,
        learning_rate=2e-4,
        output_dir="outputs",
    ),
)

trainer.train()
```

## Training with OpenAI

```python
from openai import OpenAI
client = OpenAI()

# Upload file
file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune"
)

# Create job
job = client.fine_tuning.jobs.create(
    training_file=file.id,
    model="gpt-4o-mini-2024-07-18",
    hyperparameters={
        "n_epochs": 3,
        "learning_rate_multiplier": 1.0,  # 0.1-2.0
        "batch_size": 4,
    }
)

# Monitor
while True:
    job = client.fine_tuning.jobs.retrieve(job.id)
    print(f"Status: {job.status}")
    if job.status in ["succeeded", "failed"]:
        break
    time.sleep(60)
```

## Monitoring Training

### Key Metrics to Watch

| Metric | Healthy | Problem |
|--------|---------|---------|
| Training loss | Decreasing smoothly | Jumping, not decreasing |
| Validation loss | Decreasing, then stable | Increasing = overfitting |
| Gradient norm | Stable, <10 | Exploding (>100) or vanishing (<0.001) |
| Learning rate | Following schedule | N/A |

### Overfitting Signals

1. Training loss continues dropping while val loss increases
2. Model memorizes training examples verbatim
3. Performance on held-out test set degrades

### Fixes for Overfitting

- Reduce epochs (try 1-2 instead of 3+)
- Add dropout (0.05-0.1)
- Reduce learning rate
- Add more training data
- Early stopping based on val loss

## Preventing Catastrophic Forgetting

Problem: Model loses general capabilities while learning specific task.

Solutions:
1. **Mix general data** — Include 20% general instruction data in training mix
2. **Lower learning rate** — Slow learning preserves base knowledge
3. **Regularization** — Weight decay, dropout
4. **Elastic Weight Consolidation** — Advanced technique for critical params

```python
# Example: mixing task-specific with general data
mixed_dataset = concatenate_datasets([
    task_specific_data,  # 80%
    general_instructions_data.select(range(len(task_specific_data) // 4))  # 20%
])
```


Security scan verdicts — ClawScan: clean | static analysis: clean | VirusTotal: clean | SkillSpector: clean (LOW)