WARNING: gemma-2-27b models don't run well in float16 precision.
This FLUTE-quantized model is released in bfloat16.

	Wiki	C4	PIQA	ARC-E	ARC-C	HellaSwag	Wino	Avg.
Unquantized	5.70	8.98	83.24	87.84	62.88	65.35	79.24	75.71
W4G64	5.69	9.31	82.53	86.45	59.22	64.13	78.21	74.11
W3G64	TBD	TBD	TBD	TBD	TBD	TBD	TBD	TBD

Evaluations are provided for models with learned scales.
Benchmark scores (zero-shot) are computed with lm-evaluation-harness.

Safetensors

Model size

8.1B params

Tensor type

F32

BF16

I16

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

Collection including radi-cho/gemma-2-27b-FLUTE