jeffrey-fong

/

invoker-13b

like 6

Hello, I managed to run your model on CPU by converting and quantizing to 4 bits using llama.cpp
https://github.com/ggerganov/llama.cpp

I have the following prompt loaded, but the answers do not seem to be very precise. Any idea on how to improve ?

./main -m ../invoker-13b/ggml-model-q4_0.gguf -n 128 -f functions.txt

Available Functions:

        {
          "name": "get_current_weather",
          "description": "Get the current weather in a given location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
              },
              "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
              }
            },
            "required": ["location"]
          }
        }

        {
            "name": "get_time",
            "description": "Get the current time",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. New York City, NY"
                    },
                    "format": {
                    "type": "string",
                    "enum": ["12-hour", "24-hour"]
                    }
                },
                "required": ["location"]
            }
        }

A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. The assistant calls functions with appropriate input when necessary.

USER: We are going on a vacation in Paris, France, maybe we should take an umbrella ?
ASSISTANT:

First attempt:
{"content": "That sounds like a great idea! I recommend using my `get_current_weather` function to check the weather forecast for Paris. You can specify the city and state as \"Paris, FR\" and the unit as \"celsius\". The response will include information on whether it is likely to rain or not during your stay.", "function_call": null} [end of text]

Second attempt:

```{"content": "Current weather in Paris is:\n\n- Temperature: 39\u00b0 F / 4\u00b0 C\n- Weather: Cloudy with Light Rain\n\nTo get a more detailed forecast, you can use the function get_current_weather and provide the location as Paris, France. \nYou can also specify the unit of temperature in either Celsius or Fahrenheit.\n\nIf you want to know more about the weather and climate in Paris, you can check out some