Chandrudp29/gpt2-lora-aisafety

GPT-2 LoRA Fine-Tune — AI Safety Text Generation

GPT-2 fine-tuned with LoRA (Low-Rank Adaptation) on AI safety concepts including alignment, interpretability, corrigibility, and governance.

Only 0.65% of parameters were trained (811K of 125M). The adapter is 3.25 MB — load it on top of any GPT-2 in seconds.

How to Use

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

tokenizer  = AutoTokenizer.from_pretrained("Chandrudp29/gpt2-lora-aisafety")
base_model = AutoModelForCausalLM.from_pretrained("gpt2")
model      = PeftModel.from_pretrained(base_model, "Chandrudp29/gpt2-lora-aisafety")
model.eval()

prompt = "AI alignment is the challenge of"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=80,
        temperature=0.7,
        do_sample=True,
        top_k=50,
        repetition_penalty=1.2,
        pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Training Details

Base model	gpt2 (117M params)
Adapter type	LoRA
LoRA rank	r=8
lora_alpha	16
target_modules	c_attn, c_proj
Trainable params	811,008 (0.65%)
Training examples	30 AI safety texts
Epochs	20
Learning rate	2e-4
Hardware	NVIDIA T4 (16GB)
Training time	18 seconds
Adapter size	3.25 MB

Training Topics

The model was fine-tuned on short texts covering:

• AI alignment and value alignment
• Mechanistic interpretability and circuits
• Sparse autoencoders and superposition
• Constitutional AI and RLHF
• Reward hacking and goal misgeneralization
• Corrigibility and AI governance
• Scalable oversight and debate

Example Prompts

AI alignment is the challenge of...
Mechanistic interpretability seeks to...
Constitutional AI is a method for...
Corrigibility refers to an AI system that...
The risks of advanced AI include...

Code

Training code: chandrudp29/lora-finetune

About LoRA

LoRA freezes pre-trained model weights and injects trainable low-rank matrices into each transformer layer. Instead of training all 125M parameters, only the two small adapter matrices (A × B where rank=8) are updated — reducing trainable parameters by 99.35%.

See: LoRA paper (Hu et al., 2021)