microsoft

/

phi-2

like 3.24k

Hi, I try to run the Model with chat-ui on a candle backend.

How should the chatPromptTemplate look like?
I started with following, derived from the Llama example and the description of the model card.
But it looks like, when asking the second time, the context of last answer is not used correctly.

Here is my configuration.

    {
      "name": "microsoft/phi-2",
      "description": "1.3b and 2.7b LLM with state of the art performance for <10b models",
      "websiteUrl": "https://huggingface.co/microsoft/phi-2",
      "userMessageToken": "",
      "userMessageEndToken": " [/INST] ",
      "assistantMessageToken": "",
      "assistantMessageEndToken": " </s><s>[INST] ",
      "preprompt": " ",
      "chatPromptTemplate" : "Instruct: {{#each messages}}{{#ifUser}}{{content}}\n{{/ifUser}}{{#ifAssistant}}{{content}}\n{{/ifAssistant}}{{/each}}\nOutput:",
      "endpoints": [{
            "type" : "tgi",
            "url": "http://chat-flame-backend:8080/model/phi-v2/"
            }],
      "promptExamples": [
        {
          "title": "Write an email from bullet list",
          "prompt": "As a restaurant owner, write a professional email to the supplier to get these products every week: \n\n- Wine (x10)\n- Eggs (x24)\n- Bread (x12)"
        }, {
          "title": "Code a snake game",
          "prompt": "Code a basic snake game in python, give explanations for each step."
        }, {
          "title": "Assist in a task",
          "prompt": "How do I make a delicious lemon cheesecake?"
        }
      ],
      "parameters": {
        "temperature": 0.1,
        "top_p": 0.95,
        "repetition_penalty": 1.2,
        "top_k": 50,
        "truncate": 3072,
        "max_new_tokens": 1024,
        "stop": ["</s>"]
      }
    },

Here is a conversation:

Prompt:
As a restaurant owner, write a professional email to the supplier to get these products every week: 

- Wine (x10)
- Eggs (x24)
- Bread (x12)

Answer:
Supplier,

I hope this email finds you well. I am writing to place an order for our weekly supplies of wine, eggs and bread. The quantity required is as follows: 10 bottles of wine, 24 cartons of eggs and 12 loaves of bread. Please let me know when the delivery can be made. Thank you for your continued support.

Best regards,
[Your Name]

Prompt:
 rewrite it more happy

Answer:
 Weekly Supplies Order - Wine, Eggs & Bread