einygpt
Here's the models I've trained using the transformer I wrote in einygpt. For reference they are:
- a multihead attention model replicating the model discussed in the TinyStories paper using the GPT2Tokenizer
- a multiquery attention model using the GPT2Tokenizer
- a grouped query attention model with the number of groups = 4 and using its own tokenizer
For playing with these model, you can view how they are used here