julep-ai
/

dolphin-2.9-llama3-70b-awq

Text Generation

Inference Endpoints

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Edit model card

AWQ Quantized version of cognitivecomputations/dolphin-2.9-llama3-70b.

For use with vllm and other inference engines.

Downloads last month: 262

Safetensors

Model size

11.3B params

Tensor type

FP16

·

I32

·