brian-yu-nexusflow commited on
Commit
68fca60
1 Parent(s): e4d91c9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -1
README.md CHANGED
@@ -49,12 +49,75 @@ NexusRaven is an open-source and commercially viable function calling LLM that s
49
  ## NexusRaven model usage
50
  NexusRaven accepts a list of python functions. These python functions can do anything (including sending GET/POST requests to external APIs!). The two requirements include the python function signature and the appropriate docstring to generate the function call.
51
 
52
- NexusRaven is highly compatible with langchain. See [langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/langchain_example.py). An example without langchain can be found in [non_langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/non_langchain_example.py)
53
 
54
  Please note that the model will reflect on the answer sometimes, so we highly recommend stopping the model generation at a stopping criteria of `["\nReflection:"]`, to avoid spending unnecessary tokens during inference, but the reflection might help in some rare cases. This is reflected in our langchain example.
55
 
 
 
56
  The "Initial Answer" can be executed to run the function.
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
 
59
  ## Training procedure
60
 
 
49
  ## NexusRaven model usage
50
  NexusRaven accepts a list of python functions. These python functions can do anything (including sending GET/POST requests to external APIs!). The two requirements include the python function signature and the appropriate docstring to generate the function call.
51
 
52
+ NexusRaven is highly compatible with langchain. See [langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/langchain_example.py). An example without langchain can be found in [non_langchain_example.py](https://huggingface.co/Nexusflow/NexusRaven-13B/blob/main/non_langchain_example.py).
53
 
54
  Please note that the model will reflect on the answer sometimes, so we highly recommend stopping the model generation at a stopping criteria of `["\nReflection:"]`, to avoid spending unnecessary tokens during inference, but the reflection might help in some rare cases. This is reflected in our langchain example.
55
 
56
+ More information about how to prompt the model can be found in [prompting_readme.md](prompting_readme.md).
57
+
58
  The "Initial Answer" can be executed to run the function.
59
 
60
+ ### Quickstart
61
+ You can run the model on a GPU using the following code.
62
+ ```python
63
+ # Please `pip install transformers accelerate`
64
+ from transformers import pipeline
65
+
66
+
67
+ pipeline = pipeline(
68
+ "text-generation",
69
+ model="Nexusflow/NexusRaven-13B",
70
+ torch_dtype="auto",
71
+ device_map="auto",
72
+ )
73
+
74
+ prompt_template = """
75
+ <human>:
76
+ OPTION:
77
+ <func_start>def hello_world(n : int)<func_end>
78
+ <docstring_start>
79
+ \"\"\"
80
+ Prints hello world to the user.
81
+
82
+ Args:
83
+ n (int) : Number of times to print hello world.
84
+ \"\"\"
85
+ <docstring_end>
86
+
87
+ OPTION:
88
+ <func_start>def hello_universe(n : int)<func_end>
89
+ <docstring_start>
90
+ \"\"\"
91
+ Prints hello universe to the user.
92
+
93
+ Args:
94
+ n (int) : Number of times to print hello universe.
95
+ \"\"\"
96
+ <docstring_end>
97
+
98
+ User Query: Question: {question}
99
+
100
+ Please pick a function from the above options that best answers the user query and fill in the appropriate arguments.<human_end>
101
+ """
102
+ prompt = prompt_template.format(question="Please print hello world 10 times.")
103
+
104
+ result = pipeline(prompt, max_new_tokens=100, return_full_text=False, do_sample=False)[0]["generated_text"]
105
+
106
+ # Get the "Initial Call" only
107
+ start_str = "Initial Answer: "
108
+ end_str = "\nReflection: "
109
+ start_idx = result.find(start_str) + len(start_str)
110
+ end_idx = result.find(end_str)
111
+ function_call = result[start_idx: end_idx]
112
+
113
+ print (f"Generated Call: {function_call}")
114
+ ```
115
+ This will output:
116
+ ```text
117
+ Generated Call: hello_world(10)
118
+ ```
119
+ Which can be executed.
120
+
121
 
122
  ## Training procedure
123