How to reduce GPU memory?

by ulrika-cyl - opened May 31

May 31

during my infer test, the gpu memory usage up to 70G

delian

18 days ago

me too, why the memory usage of int4 model is up to 70G?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment