Tiny Shakespeare Transformer
A small character-level causal Transformer trained from scratch on Tiny Shakespeare.
Architecture
- Character tokenizer
- 4 transformer blocks
- 4 attention heads
- embedding dimension 64
- context length 32
Loading
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"YOUR_USERNAME/tiny-shakespeare",
trust_remote_code=True
)
- Downloads last month
- 112