Esmaelmoat/SHAKESPEARE_GPT2

---

license: mit

• gpt2
• shakespeare
• causal-lm
• text-generation

model-index

• name: SHAKESPEAR_GTP2
  results (2nd line):

  First Citizen: We’ll have that end.

Full generated passage

if we have power to the people, we come not suffer us.
First Citizen: We’ll have that end.
CORIOLANUS: You are, noble Marcius, You have that come in arms.
BRUTUS: What! let’s to the gods do?
SICINIUS: That would you do?
First Citizen: I would he were well.
CORIOLANUS: I know you have heard me speak: I am a word, and I have not to do it as a man.

⚔️ SHAKESPEAR_GTP2 · Tiny GPT-2 trained from scratch on Shakespeare-style text

Model Page: Esmaelmoat/SHAKESPEAR_GTP2

Welcome! This repository hosts a tiny GPT-2 language model (8 layers, 256 hidden) trained from scratch on a corpus of Shakespearean dialogue. The model is lightweight enough to run on laptops or even Apple-Silicon devices, yet still produces convincingly archaic-English prose for creative experiments, chatbots, or educational demos.

Status: Preview Epochs trained 100 | Checkpoint size ≈ 30 MB | License MIT

---

🗂️ Resources

Resource	Link
Training script	fine_tune_gpt2_from_scratch_mps.py
Tokenizer files	see Files & versions tab
Responsible AI Toolkit	Hugging Face safe-completion docs

---

Model Information

• Architecture

Component	Value
Base	GPT-2 (from‐scratch)
Layers / Heads	8 / 8
Hidden size	256
Context length	256 tokens
Vocab size	5 000
Parameters	≈ 7.63 M
Dropout	0.1 (attn/embd/resid)

• Inputs → Outputs

Modality	Shape	Description
Text input	up to 256 tokens	Archaic/modern English prompt
Text output	1 – 256 tokens	Autoregressive continuation in Shakespearean style

---

📝 Description

SHAKESPEAR_GTP2 is a compact causal-language-model tuned exclusively on public-domain Shakespeare works (plays + sonnets). Its small footprint (≈ 30 MB weights) makes it ideal for:

• Creative writing – generate iambic pentameter lines or stage directions.
• Educational demos – illustrate tokenisation, sliding-window training, or GPT-2 internals on limited hardware.
• Chatbots or game NPCs – add Elizabethan flavour without a server-scale model.

Despite its size, the model can produce coherent Shakespeare-like sentences, though factual knowledge is limited to the training corpus (no world knowledge after 1600 CE).

---

🔧 Training Details

Setting	Value
Dataset	Public-domain Shakespeare text (Project Gutenberg) stored in one CSV column text
Total tokens	~0.9 M
Tokeniser	Byte-Level BPE (5000 vocab, min freq 3)
Window	256 tokens, 50 % overlap
Train / Val split	95 % / 5 %
Epochs	100
Optimiser	AdamW
LR schedule	Linear, peak 8 e-5, warm-up 5 %
Gradient accum	4
Hardware	Apple M-series GPU (MPS backend)
Framework	PyTorch / Transformers 4.x
Final losses	train_loss ≈ 1.92, eval_loss ≈ 1.90

---

📈 Evaluation (quick sanity)

Metric	Value
Perplexity (eval)	~6.7
Generation sample	“First Citizen: Before we proceed any further, if we have power to the people, we come not suffer us. First Citizen: We'll have that end. CORIOLANUS: You are, noble Marcius, You have that come in arms. BRUTUS: What! let's to the gods do? SICINIUS: That would you do? First Citizen: I would he were well. CORIOLANUS: I know you have heard me speak: I am a word, and I have not to do it as a man

---

💻 Usage

from transformers import GPT2LMHeadModel, GPT2TokenizerFast

model_id = "Esmaelmoat/SHAKESPEAR_GTP2"
tok  = GPT2TokenizerFast.from_pretrained(model_id)
gpt2 = GPT2LMHeadModel.from_pretrained(model_id)

prompt = "To be, or not to be"
out = gpt2.generate(
    **tok(prompt, return_tensors="pt"),
    max_length=100,
    do_sample=True,
    temperature=0.8,
    top_k=40,
    top_p=0.95,
)
print(tok.decode(out[0], skip_special_tokens=True))

---

🚦 Limitations & Risks

• Small context (256 tokens) – long-form coherence degrades.
• Archaic bias – output mimics Shakespeare; modern English or factual queries may be poor.
• Factuality – model hallucinations are common; do not rely on statements as fact.
• Training text contains historical stereotypes; generated content may reflect them.

---

👐 Intended Uses

Use-case	Examples
Creative exploration	poetry, stage dialogue, themed emails
Teaching / research	tokenisation demos, low-resource fine-tuning
Lightweight chat	NPC dialogue in indie games

Not recommended for: legal, medical, or factual Q&A.

---

📜 Citation

@misc{shakespear_gpt2_2025,
  author       = {Esmael M. Aly Shaban},
  title        = {SHAKESPEAR\_GTP2: A Tiny GPT-2 Trained on Shakespeare},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/Esmaelmoat/SHAKESPEAR_GTP2}},
  note         = {MIT License}
}

---

✉️ Contact

Questions or feedback? Create an issue on the repo or reach me on Hugging Face @Esmaelmoat.