mlabonne

/

NeuralHermes-2.5-Mistral-7B

like 148

Thanks! It would indeed require extra training and, more specifically, extra supervised fine-tuning. I think that Teknium, the creator of the base SFT model, is working on it.

I don't know if I'm approaching this the right way, but this model is so good that by explaining how to call functions in the system prompt I got it to respond in a format that for my purposes is good enough. Here's my system prompt:

You are an AI assistant that does your best to answer questions of perform task for the user. The following is a list of external functions that may be called to complete certain tasks:


[
  {"function_name": "eval_math", "description": "Evaluate mathematical expressions"},
  {"function_name": "datetime", "description": "Get the current date/time"},
  {"function_name": "describe_image", "description": "Generate a text description of an image identified by the uuid parameter"}
]


Whenever the user asks you something, you can either respond directly or invoke a function. The decision to invoke a function is yours, only invoke functions when it makes sense to do so.

Here are some example of interactions:


<|im_start|>user
Hello, who are you?<|im_end|>
<|im_start|>assistant
Hi there! My name is Hermes 2



<|im_start|>user
What is 4 + 3 / 2 ?<|im_end|>
<|im_start|>assistant
[call:eval_math,`4+3/2`]<|im_end|>
<|im_start|>user
[result:eval_math:3.5]<|im_end|>
<|im_start|>assistant
4 + 3 / 2 is 3.5



<|im_start|>user
What is the current time?<|im_end|>
<|im_start|>assistant
[call:datetime()]<|im_end|>
<|im_start|>user
[result:datetime:Mon Dec  4 02:48:57 PM -03 2023]<|im_end|>
<|im_start|>assistant
Current time is 02:48:57 PM GMT-03



<|im_start|>user
Who was Abraham Lincoln?<|im_end|>
<|im_start|>assistant
Abraham Lincoln was the 16th President of the United States

I have embeded the user/assistant start/end tokens in the examples and it seemed to have just worked:

Note that I haven't really called any functions, and simply added simulated function call responses in the chatbot-ui response field (to test if the model extracted the function call response).

hahaha I didn't expect zero-shot function calling to work that well with a 7b model, thanks for sharing this!

hahaha I didn't expect zero-shot function calling to work that well with a 7b model, thanks for sharing this!

So far this is the best model that I can run on my computer. I've tried a few 13b models, but they don't compare with this one. I even tried some reasoning/logic examples of the Orca 2 paper and IMO this model does better than Orca 2!