CUDA Error when generate

#9
by heungson - opened
This comment has been hidden
heungson changed discussion status to closed
heungson changed discussion title from CUDA error: CUBLAS_STATUS_NOT_INITIALIZED to CUDA Error when using StoppingCriteria
heungson changed discussion title from CUDA Error when using StoppingCriteria to CUDA Error when generate

When generate() is called, the model raises the following error:

CUDA error - CUBLAS_STATUS_NOT_INITIALIZED.

It seems that the error occurs due to an issue when loading the model and can sometimes be resolved by either resizing the token embeddings with

model.resize_token_embeddings(len(tokenizer))

or by reloading the model.

The model is running on GPU GV100 (32GB RAM) and I didn't add any new token to the tokenizer.

heungson changed discussion status to open

Sign up or log in to comment