brian-yu-nexusflow commited on
Commit
86ba0b1
1 Parent(s): 68fca60

Upload prompting_readme.md

Browse files
Files changed (1) hide show
  1. prompting_readme.md +172 -0
prompting_readme.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## How To Formulate The Prompt
2
+
3
+ NexusRaven-13B is instruction tuned with a structured prompt for the function calling task. To get the best use out of the model, please ensure to follow the prompt structure.
4
+
5
+ ### Prompt Template
6
+
7
+ The following is the prompt template you can use to fill in your items.
8
+
9
+ #### ChatML-Like
10
+ Despite this model being single turn only, the model uses ```<human>``` and ```<human_end>``` tags to define human sequences. It also uses ```<bot>``` and ```<bot_end>``` tags to define bot sequences. However, it is highly recommended to stop the bot at "Reflection:", as the bot is trained to reflect on its initial answer during training, which is not required during inference.
11
+
12
+ #### Defining Functions
13
+
14
+ Please use the following template to define a single function:
15
+
16
+ ```python
17
+ OPTION:
18
+ <func_start>def {function_signature}<func_end>
19
+ <docstring_start>
20
+ """
21
+ {function_docstring}
22
+ """
23
+ <docstring_end>
24
+ ```
25
+
26
+ For example, to define a hello world function, your prompt will look like this:
27
+ ```python
28
+ OPTION:
29
+ <func_start>def hello_world(n : int)<func_end>
30
+ <docstring_start>
31
+ """
32
+ Prints hello world to the user.
33
+
34
+ Args:
35
+ n (int) : Number of times to print hello world.
36
+ """
37
+ <docstring_end>
38
+ ```
39
+
40
+ We use the default pythonic way of defining the docstrings. The model should be able to learn and properly use an appropriately formatted python docstring, but it is always highly recommended to describe your function's purpose and the arguments well in the docstring.
41
+
42
+ #### Defining User Query
43
+
44
+ The first line in the following template contains the user's actual question. The second line is the instruction to the model. Please see the template below:
45
+
46
+ ```python
47
+ User Query: Question: {input}
48
+
49
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
50
+ ```
51
+
52
+ So, your query might look like this for the following user question (but please notice that the instruction does not change):
53
+ ```python
54
+ User Query: Question: Please print hello world 10 times.
55
+
56
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
57
+ ```
58
+
59
+ #### FewShot Examples
60
+
61
+ You'll notice that we have "User Query:" and "Question:". This might look redundant for zeroshot, but it serves a strong purpose for fewshot. You can include fewshot examples between the "User Query" and "Question", and the model will leverage those examples. Here is a demonstration of the prompt.
62
+
63
+ ```python
64
+ User Query: Here are some examples of similar function calls to generate. Example: How do I announce I'm here to this world someone 3 times? Answer: hello_world(3). Example: How do I tell someone helloworld 2 times? Answer: hello_world(2). Now, please answer this question. Question: Please print hello world 10 times.
65
+
66
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
67
+ ```
68
+
69
+ You can include your demonstration examples between ```User Query``` and ```Question```, and ensure you add your final question to the model after ```Question```.
70
+
71
+ ## Putting It All Together
72
+ Please start your prompt with ```<human>```.
73
+
74
+ To prompt the model in zeroshot, please do something this like:
75
+ ```python
76
+ prompt_template = """
77
+ <human>:
78
+ OPTION:
79
+ <func_start>def hello_world(n : int)<func_end>
80
+ <docstring_start>
81
+ \"\"\"
82
+ Prints hello world to the user.
83
+
84
+ Args:
85
+ n (int) : Number of times to print hello world.
86
+ \"\"\"
87
+ <docstring_end>
88
+
89
+ OPTION:
90
+ <func_start>def hello_universe(n : int)<func_end>
91
+ <docstring_start>
92
+ \"\"\"
93
+ Prints hello universe to the user.
94
+
95
+ Args:
96
+ n (int) : Number of times to print hello universe.
97
+ \"\"\"
98
+ <docstring_end>
99
+
100
+ User Query: Question: {question}
101
+
102
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
103
+ """
104
+ prompt = prompt_template.format(question="Please print hello world 10 times.")
105
+ ```
106
+
107
+ You're welcome to add an arbitrary number of functions in the same format. Using this driver code:
108
+ ```python
109
+ # Please `pip install transformers accelerate`
110
+ from transformers import pipeline
111
+
112
+
113
+ pipeline = pipeline(
114
+ "text-generation",
115
+ model="Nexusflow/NexusRaven-13B",
116
+ torch_dtype="auto",
117
+ device_map="auto",
118
+ )
119
+
120
+ result = pipeline(prompt, max_new_tokens=100, return_full_text=False, do_sample=False)[0]["generated_text"]
121
+
122
+ # Get the "Initial Call" only
123
+ start_str = "Initial Answer: "
124
+ end_str = "\nReflection: "
125
+ start_idx = result.find(start_str) + len(start_str)
126
+ end_idx = result.find(end_str)
127
+ function_call = result[start_idx: end_idx]
128
+
129
+ print (f"Generated Call: {function_call}")
130
+ ```
131
+ The call will print out:
132
+ ```text
133
+ Generated Call: hello_world(10)
134
+ ```
135
+
136
+
137
+ To prompt the model in fewshot, please do something this like:
138
+
139
+ ```python
140
+ PROMPT = \
141
+ """
142
+ <human>:
143
+ OPTION:
144
+ <func_start>def hello_world(n : int)<func_end>
145
+ <docstring_start>
146
+ \"\"\"
147
+ Prints hello world to the user.
148
+
149
+ Args:
150
+ n (int) : Number of times to print hello world.
151
+ \"\"\"
152
+ <docstring_end>
153
+
154
+ OPTION:
155
+ <func_start>def hello_universe(n : int)<func_end>
156
+ <docstring_start>
157
+ \"\"\"
158
+ Prints hello universe to the user.
159
+
160
+ Args:
161
+ n (int) : Number of times to print hello universe.
162
+ \"\"\"
163
+ <docstring_end>
164
+
165
+ User Query: Example: How do I announce I'm here to this world someone 3 times? Answer: hello_world(3). Example: How do I tell someone hello universe 2 times? Answer: hello_universe(2). Now, please answer this question. Question: Please print hello universe 14 times.
166
+
167
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>"""
168
+ ```
169
+ This code will print:
170
+ ```text
171
+ Generated Call: hello_universe(14)
172
+ ```