vuiseng9
/

ov-opt-350m-8bit-85pc-sparse-kv-cache

Text Generation

Generated from Trainer

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

ov-opt-350m-8bit-85pc-sparse-kv-cache / ov_rt_tld.json

vuiseng9's picture

Add tld config and guide

7d6d1c1 8 months ago

raw history blame contribute delete

No virus

79 Bytes

	{
	"CPU": {
	"CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE": "0.50"
	}
	}