xzuyn/GPT-2-Small-Stripped

This is just an experiment to see if a script of mine works.

GPT-2-Small, but I removed all layers in between h.0.mlp.c_proj.bias and ln_f.weight.

These are the only layers/tensors left (I'm unsure of the terminology for these):

wte.weight
wpe.weight
h.0.ln_1.weight
h.0.ln_1.bias
h.0.attn.bias
h.0.attn.c_attn.weight
h.0.attn.c_attn.bias
h.0.attn.c_proj.weight
h.0.attn.c_proj.bias
h.0.ln_2.weight
h.0.ln_2.bias
h.0.mlp.c_fc.weight
h.0.mlp.c_fc.bias
h.0.mlp.c_proj.weight
h.0.mlp.c_proj.bias
ln_f.weight
ln_f.bias