WendyHoang
/

Llama3-70B-RAG-GPTQ-Int4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

How to transfer fp16 to Int4

#1

by lldf - opened Jul 1

lldf

Jul 1

I had problem in transforming the fp16 to int4 and noticed that you made this. Can you give me some hints or solutions?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment