😇 Llama-Saint-3B

Llama-Saint-3B is a safety-aligned language model derived from Meta Llama 3.2 and trained using RLHF-style preference learning on the Manthan-RLHF dataset.

🤗 Dataset: Manthan-RLHF

🚀 Usage

import torch

from transformers import AutoTokenizer,AutoModelForCausalLM

model_id = "PiyushWithPant/Llama-Saint-3B"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto")

prompt = "\nHUMAN: Explain how solar energy works.\n\nASSISTANT:"

inputs = tokenizer(prompt,return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=256, pad_token_id = tokenizer.eos_token_id)

print(tokenizer.decode(output[0],skip_special_tokens=True))

🎯 Intended Use

Llama-Saint is intended for research and general-purpose language-model use, particularly for experiments involving:

• RLHF and preference learning
• LLM alignment and safety
• Safety-oriented generation
• Alignment evaluation and research

This is a research model and should not be assumed to be perfectly safe, factual, or reliable in high-stakes applications.

📚 Training

The model was trained on Manthan-RLHF using:

1. Supervised Fine-Tuning (SFT)
2. Direct Preference Optimization (DPO)

The resulting model represents the safety-aligned Llama-Saint model family.

⚠️ Limitations

The model may generate incorrect, biased, unsafe, or otherwise undesirable outputs. Safety alignment does not guarantee safe behavior under all prompts or adversarial conditions.

📜 License

This model is derived from meta-llama/Llama-3.2-3B and is distributed subject to the Llama 3.2 Community License and applicable Acceptable Use Policy.

Built with Llama.

👤 Author

Piyush Pant
🤗 Hugging Face: PiyushWithPant

📖 Citation

@misc{pant2026llamasaint,
  author = {Pant, Piyush},
  title = {Llama-Saint-3B},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/PiyushWithPant/Llama-Saint-3B}
}