imiraoui
/

OpenHermes-2.5-Mistral-7B-sharded

Very Slow Generation on google colab

by delitante-coder - opened Nov 20, 2023

Nov 20, 2023

I am loading model in 4 bit, and also using bnd_compute_dtype = torch.bfloat16.
Is anyone else face the same issue ?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment