meta-llama

/

LlamaGuard-7b

like 175

Hi there,

I am having trouble meeting the same latency after training the model in bfloat16 (and other dtypes I assume).
Does LlamaGuard-7b have any tricks to make inference faster that could potentially lost after uploading a fine-tuned version of it to the hub ?

Thanks,
James