How much VRAM it requires for openthaigpt/openthaigpt1.5-14b-instruct model?

by ChirawatCh - opened Oct 14, 2024

Discussion

ChirawatCh

Oct 14, 2024

I wan to know the mininum VRAM it require?

Thaweewat

OpenThaiGPT org Oct 16, 2024

OpenThaiGPT 1.5 14B is based on Qwen 2.5 14B, which requires at least 29.6 GB of VRAM as suggested by doc (Reference) However, with the quantized version, you can reduce this requirement at the cost of throughput. Someone has done a benchmark here.

thisnamealreadyExists

19 days ago

1.6 มาตอนไหนหรอครับ

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment