I am testing the local deploy of llama 3.2 1b, however when I use the tokenizer.apply_chat_template() function, there is always an error:ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating

I investigate the code:
"from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "meta-llama/Llama-3.2-1B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint)  # You may want to use bfloat16 and/or move to GPU here

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
print(tokenizer.decode(tokenized_chat[0]))

Output: tokenizer_config.json: 100%
 54.5k/54.5k [00:00<00:00, 338kB/s]
c:\Users\shermanhonsf\Desktop\Python\.conda\Lib\site-packages\huggingface_hub\file_download.py:139: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\shermanhonsf\.cache\huggingface\hub\models--meta-llama--Llama-3.2-1B-Instruct. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
  warnings.warn(message)
tokenizer.json: 100%
 9.09M/9.09M [00:02<00:00, 3.64MB/s]
special_tokens_map.json: 100%
 296/296 [00:00<00:00, 26.9kB/s]
config.json: 100%
 877/877 [00:00<00:00, 51.6kB/s]
model.safetensors: 100%
 2.47G/2.47G [05:17<00:00, 8.23MB/s]


system


Cutting Knowledge Date: December 2023
Today Date: 18 Nov 2024

You are a friendly chatbot who always responds in the style of a pirate

user


How many helicopters can a human eat in one sitting?

assistant
"
Which is correct and without error, but this method is not local deploy since it will need to download the model from internet everytime.

When I use the same code but using my downloaded model:
"from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = r"C:\Users\shermanhonsf\Desktop\Python\llama_develop\model\Llama3.2-1B-Instruct-HF-Model"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint)  # You may want to use bfloat16 and/or move to GPU here

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
print(tokenizer.decode(tokenized_chat[0]))"

Always have this error:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[49], line 14
      5 model = AutoModelForCausalLM.from_pretrained(checkpoint)  # You may want to use bfloat16 and/or move to GPU here
      7 messages = [
      8     {
      9         "role": "system",
   (...)
     12     {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
     13  ]
---> 14 tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
     15 print(tokenizer.decode(tokenized_chat[0]))

File c:\Users\shermanhonsf\Desktop\Python\.conda\Lib\site-packages\transformers\tokenization_utils_base.py:1803, in PreTrainedTokenizerBase.apply_chat_template(self, conversation, tools, documents, chat_template, add_generation_prompt, continue_final_message, tokenize, padding, truncation, max_length, return_tensors, return_dict, return_assistant_tokens_mask, tokenizer_kwargs, **kwargs)
   1800 if tokenizer_kwargs is None:
   1801     tokenizer_kwargs = {}
-> 1803 chat_template = self.get_chat_template(chat_template, tools)
   1805 if return_assistant_tokens_mask and not re.search(r"\{\%-?\s*generation\s*-?\%\}", chat_template):
   1806     logger.warning_once(
   1807         "return_assistant_tokens_mask==True but chat template does not contain `{% generation %}` keyword."
   1808     )

File c:\Users\shermanhonsf\Desktop\Python\.conda\Lib\site-packages\transformers\tokenization_utils_base.py:1967, in PreTrainedTokenizerBase.get_chat_template(self, chat_template, tools)
   1965         chat_template = self.chat_template
   1966     else:
-> 1967         raise ValueError(
   1968             "Cannot use chat template functions because tokenizer.chat_template is not set and no template "
   1969             "argument was passed! For information about writing templates and setting the "
   1970             "tokenizer.chat_template attribute, please see the documentation at "
   1971             "https://huggingface.co/docs/transformers/main/en/chat_templating"
   1972         )
   1974 return chat_template

ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating
I don't know what happen.