hugging-quants
/

Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Resources

View closed (9)

Interview request: genAI evaluation & documentation

#16 opened about 2 months ago by

add base_model

#15 opened 2 months ago by

add base_model metadata

#14 opened 3 months ago by

Issues to set the seed

#13 opened 3 months ago by

could not run this in lmdeploy

#12 opened 3 months ago by

How to use INT4 with vLLM?

#8 opened 4 months ago by

Fixed tokenizer

#6 opened 4 months ago by