GitHub repository: loftusa/GPTNext
Primary language: Python
Description: Starting from Andrej Karpathy's NanoGPT repo, progressively introduce the various improvements to transformers that were developed over the last four years, and see which one gets the most "bang-for-buck". Inspired by the ConvNEXT paper.