unsubscribe commited on
Commit
8fa9dff
1 Parent(s): 2798175

add function call example

Browse files
Files changed (1) hide show
  1. README.md +45 -0
README.md CHANGED
@@ -55,6 +55,8 @@ huggingface-cli download internlm/internlm2_5-7b-chat-gguf internlm2_5-7b-chat-f
55
 
56
  You can use `llama-cli` for conducting inference. For a detailed explanation of `llama-cli`, please refer to [this guide](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
57
 
 
 
58
  ```shell
59
  build/bin/llama-cli \
60
  --model internlm2_5-7b-chat-fp16.gguf  \
@@ -76,6 +78,49 @@ build/bin/llama-cli \
76
  --in-suffix "<|im_end|>\n<|im_start|>assistant\n"
77
  ```
78
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79
  ## Serving
80
 
81
  `llama.cpp` provides an OpenAI API compatible server - `llama-server`. You can deploy `internlm2_5-7b-chat-fp16.gguf` into a service like this:
 
55
 
56
  You can use `llama-cli` for conducting inference. For a detailed explanation of `llama-cli`, please refer to [this guide](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
57
 
58
+ ### chat example
59
+
60
  ```shell
61
  build/bin/llama-cli \
62
  --model internlm2_5-7b-chat-fp16.gguf  \
 
78
  --in-suffix "<|im_end|>\n<|im_start|>assistant\n"
79
  ```
80
 
81
+ ### Function call example
82
+
83
+ `llama-cli` example:
84
+
85
+ ```shell
86
+ build/bin/llama-cli \
87
+ --model internlm2_5-7b-chat-fp16.gguf \
88
+ --predict 512 \
89
+ --ctx-size 4096 \
90
+ --gpu-layers 32 \
91
+ --temp 0.8 \
92
+ --top-p 0.8 \
93
+ --top-k 50 \
94
+ --seed 1024 \
95
+ --color \
96
+ --prompt '<|im_start|>system\nYou are InternLM2-Chat, a harmless AI assistant.<|im_end|>\n<|im_start|>system name=<|plugin|>[{"name": "get_current_weather", "parameters": {"required": ["location"], "type": "object", "properties": {"location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"}, "unit": {"type": "string"}}}, "description": "Get the current weather in a given location"}]<|im_end|>\n<|im_start|>user\n' \
97
+ --interactive \
98
+ --multiline-input \
99
+ --conversation \
100
+ --verbose \
101
+ --in-suffix "<|im_end|>\n<|im_start|>assistant\n" \
102
+ --special
103
+ ```
104
+
105
+ Conversation results:
106
+
107
+ ```text
108
+ <s><|im_start|>system
109
+ You are InternLM2-Chat, a harmless AI assistant.<|im_end|>
110
+ <|im_start|>system name=<|plugin|>[{"name": "get_current_weather", "parameters": {"required": ["location"], "type": "object", "properties": {"location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"}, "unit": {"type": "string"}}}, "description": "Get the current weather in a given location"}]<|im_end|>
111
+ <|im_start|>user
112
+
113
+ > I want to know today's weather in Shanghai
114
+ I need to use the get_current_weather function to get the current weather in Shanghai.<|action_start|><|plugin|>
115
+ {"name": "get_current_weather", "parameters": {"location": "Shanghai"}}<|action_end|>
116
+ <|im_end|>
117
+
118
+ > <|im_start|>environment name=<|plugin|>\n{"temperature": 22}
119
+ The current temperature in Shanghai is 22 degrees Celsius.<|im_end|>
120
+
121
+ >
122
+ ```
123
+
124
  ## Serving
125
 
126
  `llama.cpp` provides an OpenAI API compatible server - `llama-server`. You can deploy `internlm2_5-7b-chat-fp16.gguf` into a service like this: