Added example to run on smaller GPUS

#1

Current setup would not run on A10's, so added an example that would allow it to run in on T4 (16GB gpus)

Norwegian Large Language Models org

Hi, thank you very much, this is very useful information for a lot of people :)

davda54 changed pull request status to merged

@davda54 If you are interested, I'll see if I get time to split the model into three shards and convert them to safetensor. Doing this would most likely enable loading it in a free Colab environment. (Or you can ofc do it yourself if :) )

Norwegian Large Language Models org

That's a great idea, I added sharded safetensor weights to all three repositories, now they run in a free-tier Colab notebook. Thanks!

Sign up or log in to comment