How much VRAM is required? I have 8gb RTX 3060 and seems in sufficient

#20

by LawItApps - opened 9 days ago

9 days ago

Hey all

I am running this model according to the instructions but it keeps saying I am running out of VRAM despite having 8gb with my RTX 3060.

Seems abnormal as the model is roughly 6gb.

Can anyone help with this? As I don't think I have sufficient VRAM to run the AWQ 7b model.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment