有出量化模型INT4的计划吗

#21

by Winkuis - opened 9 days ago

9 days ago

有出量化模型INT4的计划吗，非量化太大了，机器跑不动，想尝试一下

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org 7 days ago

我们有提供load in 4bit，如果你的内存没办法做到以load in 4bit读入，魔搭社区有社区直接把load in 4bit 导出的4bit模型

zRzRzRzRzRzRzR changed discussion status to closed 7 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment