pankajroark
/

llama-fp16-engine

Model card Files Files and versions Community

llama-fp16-engine / 7b-sq-int8kv-tp8 /llama_float16_tp8_rank4.engine

Commit History

update engine with inflight options for 7b-sq-int8kv-tp8

ad987a1

pankajroark commited on Oct 26, 2023

tp8 checkpoint

a3806a1

pankajroark commited on Oct 24, 2023