Package: agentools
AgenTools- Async Generator Tools for LLMsA simple set of modules, wrappers and utils that are essential for LLM-based assistants and agents using the OpenAI API and function tools. It is useful for:OpenAI API:Simple wrapper for the OpenAI API to provide mocked endpoints for easy testing without costing money, accumulating the delta chunks from streamed responses into partial responses, and easier token counting/tracking.Function Tools:Easily convert any (async) python function into a function tool that the LLM model can call, with automatic validation and retrying with error messages.Structured Data:Easily define a Pydantic model that can be generated by the LLM model, also with validation and retries.Assistants:Event-based architecture with async generators that yield events that you can iterate through and handle only the events you care about, such as whether you want to stream the response or not, cancel the generation prematurely, or wait for user input (human-in-the-loop) before continuing, etc.Copilots:Integrate right into an editor with stateful system messages to allow the copilot to see the latest state of the editor and function tools to interact with the editor.Yet to come:Agents:Autoprompting, self-prompting, chain-of-thought, sketchpads, memory management, planning, and more.Multi-Agents: Communication channels, organization structuring, and more.Quick StartInstallationpipinstallagentoolsAssistant and ChatGPTA high-level interface to use ChatGPT or other LLM-based assistants! The default implementation of ChatGPT has:a message history to remember the conversation so far (including the system prompt)ability to use toolsefficient async streaming supportsimple way to customize/extend/override the default behaviorfromagentoolsimport*# empty chat history and default model (gpt-3.5)model=ChatGPT()You can then simply call the model as if it was a function, with a prompt:awaitmodel("Hey!")'Hello! How can I assist you today?'As you can see, the model is async and it simply returns the resonse as a string.Both your prompt and the response are stored in the history, so you can keep calling the model with new prompts and it will remember the conversation so far.awaitmodel("Can you repeat my last message please?")'Of course! Your last message was: "Hey!"'model.messages.history[{'role': 'user', 'content': 'Hey!'},{'content': 'Hello! How can I assist you today?', 'role': 'assistant'},{'role': 'user', 'content': 'Can you repeat my last message please?'},{'content': 'Of course! Your last message was: "Hey!"', 'role': 'assistant'}]System prompt and more onMessageHistoryNotice that our model has no system prompt in the beginning.ChatGPT's constructor by default creates an empty chat history, but you can explicitly create aMessageHistoryobject and pass it to the constructor:translate=ChatGPT(messages=SimpleHistory.system("Translate the user message to English"))# SimpleHistory.system(s) is just shorthand for SimpleHistory([msg(system=s)])print(awaittranslate("Ich liebe Katzen!"))print(awaittranslate("고양이랑 강아지 둘다 좋아!"))I love cats!I like both cats and dogs!translate.messages.history[{'role': 'system', 'content': 'Translate the user message to English'},{'role': 'user', 'content': 'Ich liebe Katzen!'},{'content': 'I love cats!', 'role': 'assistant'},{'role': 'user', 'content': '고양이랑 강아지 둘다 좋아!'},{'content': 'I like both cats and dogs!', 'role': 'assistant'}]Notice that here, we're wasting tokens by remembering the chat history, since it's not really a conversation. There's a simpleGPTclass, which simply resets the message history after each prompt:translate=GPT(messages=SimpleHistory.system("Translate the user message to English"))awaittranslate("Ich liebe Katzen!")awaittranslate("고양이랑 강아지 둘다 좋아!")translate.messages.history[{'role': 'system', 'content': 'Translate the user message to English'}]OpenAI API: changing the model and mocked APIYou can set the default model in the constructor, or override it for each prompt:# default model is now gpt-4 💸model=ChatGPT(model="gpt-4")# but you can override it for each prompt anywaysawaitmodel("Heyo!",model="mocked")'Hello, world!'As you see, our wrapper provides a simple mocked "model", which will simply return"Hello, world!"for any prompt, with some simulated latency. This will also work with streaming responses, and in either cases, you won't be able to tell the difference between the real API and the mocked one.There are more mocked models for your convinience:mocked: always returns"Hello, world!"mocked:TEST123: returns the string after the colon, e.g."TEST123"echo: returns the user prompt itselfLet's print all events to the console to take a peek at the event-based generator:awaitmodel("Heya!",model="echo",event_logger=print)[ResponseStartEvent]: prompt=Heya!, tools=None, model=echo, max_function_calls=100, openai_kwargs={}[CompletionStartEvent]: call_index=0[CompletionEvent]: completion=ChatCompletion(id='mock', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Heya!', role='assistant', function_call=None, tool_calls=None))], created=1706671307, model='mock', object='chat.completion', system_fingerprint=None, usage=None), call_index=0[FullMessageEvent]: message=ChatCompletionMessage(content='Heya!', role='assistant', function_call=None, tool_calls=None), choice_index=0[TextMessageEvent]: content=Heya![ResponseEndEvent]: content=Heya!'Heya!'Wow, quite a lot going on for a simple prompt! While it might seem like too many events, this offers a lot of flexibility and customizability.You can easily handle only the events you are interested in, useful when e.g:updating the frontend when streaming the responses,cancelling the generation early,or implementing human-in-the-loop for function calls.For instance, theGPTclass from above is as simple as:asyncforeventinself.response_events(prompt,**openai_kwargs):matchevent:caseself.ResponseEndEvent():awaitself.messages.reset()returnevent.contentThis generator-based architecture is a good balance between flexibility and simplicity!While we won't go deeper into the low-level API in this quickstart, you can look at theadvanced.ipynbnotebook for more details.Tools:@function_toolYou can turn any function into a tool usable by the model by decorating it with@function_tool:@function_tooldefprint_to_console(text:str)->str:"""Print text to consoleArgs:text: text to print"""print(text)return"success"# the model will see the return value# normal callprint_to_console("Hello from python!")Hello from python!'success'You can use the tool from python as you normally would, and the model will also be able to use it simply by passing it to thetoolsparameter during init (as default) or prompting it (as a one-off).model=ChatGPT(tools=print_to_console)awaitmodel("Say 'hello from GPT' to console!")Hello from GPT!'I have successfully printed "Hello from GPT!" to the console.'To make the function a@function_tool, you must do the following:The parameters must be type annotated, and all parameters must be JSON-serializable (e.g.str,int,float,bool,list,dict,None, etc).The return type should be astror something that can be converted to astr.It must be documented with a'''docstring''', including each parameter (mostformats supported, e.g.Google-style,NumPy-style, sphinx-style, etc, seethis overview)Showing off some more goodies:Even async functions should seamlessly work, just don't forget toawaitthem.@fail_with_message(err)is a decorator that will catch any exceptions thrown by the function and instead return the error message. This is useful for when you want to handle errors in a more graceful way than just crashing the model. It also takes an optional logger, which by default takes theprintfunction, but any callable that takes a string will work, such aslogger.errorfrom theloggingmodule.Usually, the@function_tooldecorator will throw an assertion error if you forget to provide the description for any of the function or their parameters. If you really don't want to provide descriptions for some (or all), maybe because it's so self-explanatory or you need to save tokens, then you can explicitly turn off the docstring parsing by passing@function_tool(check_description=False). This is not recommended, but it's there if you need it.Note that by returning descriptive error strings, the model can read the error message and retry, increasing the robustness!importasyncioimportlogging@function_tool(name="Fibonacci",require_doc=False)@fail_with_message("Error",logger=logging.error)asyncdeffib(n:int):ifn<0:raiseValueError("n must be >= 0")ifn<2:returnnawaitasyncio.sleep(0.1)returnsum(awaitasyncio.gather(fib(n-1),fib(n-2)))awaitfib(-10)ERROR:root:Tool call fib(-10) failed: n must be >= 0'Error: n must be >= 0'Toolkits:class ToolkitToolkits are a collection of related function tools, esp. useful when they share a state. Also good for keeping the state bound to a single instance of the toolkit, rather than a global state.
To create a toolkit, simply subclassToolkitand decorate its methods with@function_tool.classNotepad(Toolkit):def__init__(self):super().__init__()self.content="<Fill me in>"@function_tooldefwrite(self,text:str):"""Write text to the notepadArgs:text: The text to write"""self.content=text@function_tool(require_doc=False)defread(self):returnself.contentnotes=Notepad()notes.write("Shhh... here's a secret: 42")notes.read()"Shhh... here's a secret: 42"As before, simply pass the toolkit to the model. To use multiple tools and toolkits, simply use theToolListclass:gpt=ChatGPT(tools=ToolList(notes,print_to_console,fib),)awaitgpt("What's on my notepad?")'On your notepad, there is a secret written: 42.'awaitgpt("Can you calculate the 8th fibonacci number, add it to the number in my notes, and write it? also print it to console as well.",event_logger=lambdax:print(x)ifx.startswith("[Tool")elseNone,)[ToolCallsEvent]: tool_calls=[ChatCompletionMessageToolCall(id='call_Uj8VIxhCxvHUnbIH1CK50Qmo', function=Function(arguments='{\n  "n": 8\n}', name='Fibonacci'), type='function')][ToolResultEvent]: result=21, tool_call=ChatCompletionMessageToolCall(id='call_Uj8VIxhCxvHUnbIH1CK50Qmo', function=Function(arguments='{\n  "n": 8\n}', name='Fibonacci'), type='function'), index=0[ToolCallsEvent]: tool_calls=[ChatCompletionMessageToolCall(id='call_0F0alwZitumJTU3QlPxuoJNW', function=Function(arguments='{\n  "text": "63"\n}', name='write'), type='function')][ToolResultEvent]: result=None, tool_call=ChatCompletionMessageToolCall(id='call_0F0alwZitumJTU3QlPxuoJNW', function=Function(arguments='{\n  "text": "63"\n}', name='write'), type='function'), index=0[ToolCallsEvent]: tool_calls=[ChatCompletionMessageToolCall(id='call_KoSyY1LZQYVfEpMDPUZlt2pc', function=Function(arguments='{\n  "text": "63"\n}', name='print_to_console'), type='function')]63[ToolResultEvent]: result=success, tool_call=ChatCompletionMessageToolCall(id='call_KoSyY1LZQYVfEpMDPUZlt2pc', function=Function(arguments='{\n  "text": "63"\n}', name='print_to_console'), type='function'), index=0'I have calculated the 8th Fibonacci number to be 21. I have added it to the number on your notepad, which is 42, and the result is 63.\n\nI have written the result, 63, on your notepad. However, it seems that there was an issue with writing it. My apologies for the inconvenience.\n\nI have also printed the result, 63, to the console.'notes.read()'63'Notice how since ourwritefunction doesn't return anything, it defaults toNoneand our model gets confused! So don't forget to return an encouraging success message to make our model happy :)Structured DataWe can very easily define a Pydantic model that can be generated by the LLM model, with validation and retries:frompydanticimportBaseModel,FieldclassSong(BaseModel):title:strgenres:list[str]=Field(description="AT LEAST 3 genres!")duration:float# normal useSong(title="Hello",genres=["pop"],duration=3.5)Song(title='Hello', genres=['pop'], duration=3.5)Create aStructGPTobject with your pydantic model, and prompting it will always return a valid instance of the model, or raise an exception if it fails to generate a valid instance after the maximum number of retries. Your docstring and field descriptions will also be visible to the model, so make sure to write good descriptions!generate_song=StructGPT(Song)awaitgenerate_song("Come up with an all-time best K-hiphop song")Song(title='Respect My Name', genres=['Hip-Hop', 'K-Rap', 'Trap'], duration=240.0)Misc.Streaming can be enabled as usual by passingstream=Truewhen prompting, and handle the partial events as they come in. Check theAssistantclass for a list of events including the ones for streaming.There are some other useful utilities in theutilsmodule, such as:tokens: for token countingtrackers: for transparent token tracking and prompt/response logging