YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
trtllm-build --checkpoint_dir ./trt-ckpt/ --gemm_plugin bfloat16 --gpt_attention_plugin bfloat16 --max_batch_size 64 --max_input_len 3000 --max_output_len 1000 --context_fmha enable --output_dir ./engines
quantized to int8 as per the config.json
- Downloads last month
- 2
Unable to determine this model’s pipeline type. Check the
docs
.