vLLM help pls :(
3
#6 opened 10 days ago
by
fsaudm
How much cuda memory is needed to run this model?
2
#5 opened 17 days ago
by
JohnnyBoyzzz
Any chance of an int4 or quantised version?
3
#3 opened 18 days ago
by
smcleod