Text Generation
Transformers
PyTorch
mpt
Composer
MosaicML
llm-foundry
custom_code
text-generation-inference

Add eos_token_id to generation config

#65
by udhavsethi - opened

If the model is used without pipeline, the model shows unexpected behaviour and generates extra unneeded tokens. The eos_token_id has to be passed explicitly to fix it since the generation config does not have it.

sam-mosaic changed pull request status to merged

Sign up or log in to comment