This is a very basic pyTorch transformer model that sorts lists of numbers. It was trained with nanoGPT.
The context window is 256 tokens, so the input list can be up to 127 tokens long. Numbers can be 0 to 99, separated by comma tokens.
It was trained for about one day on a laptop with a single NVIDIA RTX 2070 eGPU, so don't expect anything amazing. In practice it sorts these lists correctly about 90% of the time, which is good enough to satisfy my curiosity.
To run, I recommend cloning nanoGPT (https://github.com/karpathy/nanoGPT) and installing its prerequisites. Create a new branch and copy these files into the nanoGPT folder, overwriting the included sample.py and train.py.
To run:
python sample.py --out_dir=out-sort-lists --start="(5,4,3,2,1): [" --num_samples=1 --temperature=0.0001 --max_new_tokens=127
To train:
python train.py config/train_sort.py