Post
3310
🔮 GPT-3 implemented in pure Free Pascal!
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
Clean Pascal Implementation
https://github.com/joaopauloschuler/gpt-3-for-pascal
This implementation follows the GPT-3 Small architecture from the landmark paper "Language Models are Few-Shot Learners":
┌─────────────────────────┐
│ Input Layer │
├─────────────────────────┤
│ Token & Positional │
│ Embedding │
├─────────────────────────┤
│ 12x Transformer │
│ Blocks │
│ - 12 heads │
│ - 768 hidden dims │
│ - 3072 intermediate │
├─────────────────────────┤
│ Output Layer │
└─────────────────────────┘Clean Pascal Implementation
for CntLayer := 1 to {Layers=}12 do
begin
Result.AddTransformerBlockCAI(
{Heads=}12,
{intermediate dimensions=}4*768,
{NoForward=}true,
{HasNorm=}true,
false
);
end;