Minerva-300M-Instruct (Experimental)

Minerva-300M is an experimental ~300 million parameter Large Language Model trained completely from scratch on a single consumer GPU (NVIDIA RTX 3060 Ti).

This model serves as a proof-of-concept and a portfolio project to demonstrate the end-to-end LLM training pipeline, including:

1. Custom Tokenizer training (BPE / minerva-tokenizer)
2. Pre-training on ~1.5 Billion tokens (interleaved Wikipedia-TR and FineWeb-Edu)
3. Instruction Fine-Tuning using TFLai/Turkish-Alpaca dataset.

⚠️ Disclaimer

This is an educational/toy model. Because it was trained on only ~1.5 Billion tokens (compared to trillions for modern LLMs) for a short duration, the model lacks the "world knowledge" and linguistic mastery required to answer questions properly. It will frequently hallucinate and produce incoherent text. It is published purely for structural demonstration and portfolio purposes to showcase pipeline engineering, not for production use.

Model Details

• Architecture: LLaMA-based (Causal LM)
• Parameters: ~300 Million
• Hidden Size: 1024
• Layers: 20
• Attention Heads: 16
• Context Length: 512
• Training Hardware: 1x NVIDIA RTX 3060 Ti (8GB VRAM)
• Optimization: FP16 Mixed Precision, Gradient Accumulation

How to Use (Python)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "your-hf-username/minerva-300m-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "Talimat: Merhaba, nasılsın?\n\nCevap: "
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=50, repetition_penalty=1.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))