How to run in Google Colab?
Hi all,
I am writing today regarding an issue I encountered while attempting to run the "OpenAssistant/oasst-sft-1-pythia-12b" model in Google Colab using the provided template from Hugging Face. I used the following code:
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")
model = AutoModelForCausalLM.from_pretrained("OpenAssistant/oasst-sft-1-pythia-12b")
While the model was loading, I encountered a "session crashed" message due to Google Colab running out of memory.
I am writing to ask if anyone has suggestions for possible workarounds that would allow me to run this model within Google Colab, or if there are alternative approaches to loading the model that would be more feasible given the resource constraints of Colab.
Thank you in advance for your assistance.
Best regards, Martin