Hugging Face's logo

codellama
/
CodeLlama-70b-Instruct-hf

Text Generation
Transformers
PyTorch
Safetensors
code
llama
llama-2
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
25

fix: prefer `| trim` over `.strip()`in tokenizer_config

#21
by drbh HF staff - opened
base: refs/heads/main
←
from: refs/pr/21
Discussion Files changed
+1
-1
drbh
•
edited Feb 14

this PR simply replaces .strip() with trim to align with standard jinja: https://jinja.palletsprojects.com/en/2.11.x/templates/#list-of-builtin-filters

fix: prefer `| trim` over `.strip()`in tokenizer_config08149a7f
pcuenq
 Code Llama org

Thank you! Tested locally, works for me. cc @Rocketknight1 in case he sees some incompatibility.

pcuenq
 Code Llama org

Also cc @Xenova for info (.strip() not supported by TGI so changing to | strip, not sure if this will be needed in more models and whether transformers.js would work with it).

Xenova

Thanks for the ping @pcuenq ! Filters (| operator) are supported by @huggingface/jinja, but trim has not yet been added yet. Luckily, it’s really simple, and I’ll open a PR for it tomorrow.

Xenova
•
edited Feb 15

@huggingface/jinja v0.1.3 is now out which adds support for | trim (link). Feel free to merge :)

pcuenq
 Code Llama org

Thanks all, merging!

pcuenq changed pull request status to merged

Sign up or log in to comment