rajammanabrolu
/

gpt-4-chat

Inference Endpoints

Model card Files Files and versions Community

rajammanabrolu commited on Jan 11, 2024

Commit

fd5557b

·

verified ·

1 Parent(s): 385df1b

Update tiktoken.py

Files changed (1) hide show

tiktoken.py +1 -1

tiktoken.py CHANGED Viewed

@@ -192,7 +192,7 @@ class TiktokenTokenizerWrapper(PreTrainedTokenizer):
             '{% for message in loop_messages %}'
             '{% if loop.index0 == 0 %}'
             '{% if system_message != false %}'
-            "{{ '<|im_start|>system\n' + system_message.strip() + '\n'}}"
             '{% endif %}'
             "{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' }}"
             '{% else %}'

             '{% for message in loop_messages %}'
             '{% if loop.index0 == 0 %}'
             '{% if system_message != false %}'
+            "{{ '<|im_start|>system\n' + system_message.strip() + '<|im_end|>\n'}}"
             '{% endif %}'
             "{{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' }}"
             '{% else %}'