Package: tokreate
TokreateA minimal library to create tokens using LLMs.I refuse to write more than 200 lines of code for this.NoLangChainshenanigans.
Seetokreate.pyto see how well I'm sticking to this.Tokreate isavailable on PyPI! Simply run:pipinstalltokreateto get started.Use the following shortcut to navigate to a section and learn more about tokreate:UsageDesignSupported APIsGitHubUsageBelow, I show some ways tokreate can be used to prompt LLMs.A Simple Call ActionfromtokreateimportCallActiongreetings=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="gpt-3.5-turbo")# tokreate always returns a history of the conversationhistory=greetings.run(name="Alice")forturninhistory:print(repr(turn))# user>>> Hello, my name is Alice; what's yours?# assistant>>> Hello Alice, nice to meet you! I'm ChatGPT. How can I assist you today?Chaining ActionsfromtokreateimportCallActiongreetings=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="gpt-3.5-turbo")tell_joke=CallAction(prompt="Can you finish this joke: What do you call a {{ animal }} with no legs?",model="gpt-3.5-turbo")history=(greetings>>tell_joke).run(name="Alice",animal="snake")forturninhistory:print(repr(turn))# user>>> Hello, my name is Alice; what's yours?# assistant>>> Hello Alice, nice to meet you! I'm ChatGPT. How can I assist you today?# user>>> Can you finish this joke: What do you call a snake with no legs?# assistant>>> What do you call a snake with no legs? A "legless" reptile!You can use parsers to extract information from any call:importjsonfromtokreateimportCallAction,ParseActionrandom_numbers=CallAction(prompt="Return a list of {{ random_count }} random numbers.",system="You are an helpful assistant who responds by always returning a valid JSON object.",model="gpt-3.5-turbo")json_parser=ParseAction(parser=json.loads)*_,last_turn=(random_numbers>>json_parser).run(random_count='five')parsed_content=last_turn.state["json.loads"]print(f"Response{parsed_content}(type:{type(parsed_content)})")# Response {'numbers': [4, 9, 2, 7, 1]} (type: <class 'dict'>)Using Different ModelsThis is an example on how to use Anthropic's models:fromtokreateimportCallActiongreetings=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="claude-instant-v1.1")history=greetings.run(name="Alice")forturninhistory:print(repr(turn))# user>>> Hello, my name is Alice; what's yours?# assistant>>> I'm Claude, an AI assistant created by Anthropic.Let's try with one of the TogetherAI models:fromtokreateimportCallActiongreetings=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="togethercomputer/llama-2-70b-chat",history=False# TogetherAI models don't support history at API level yet :( you can make your own in prompt)history=greetings.run(name="Alice")forturninhistory:print(repr(turn))# user>>> Hello, my name is Alice; what's yours?# assistant>>>  Hello Alice! My name is ChatBot, it's nice to meet you. How can I assist you today? Is there a specific topic you'd like to discuss or ask me a question about?Async SupportYou can make calls to LLMs asynchronously thanks to async support in tokreate. Note that some APIs (e.g., Anthropic) might support very few concurrent calls by default.importasynciofromtokreateimportCallActionasyncdefmain():greetings=CallAction(prompt="Generate a very short description of the color {{ color }}.",model="gpt-3.5-turbo")futures=[]forcolorin["red","green","blue"]:futures.append(greetings.arun(color=color))results=awaitasyncio.gather(*futures)for(request,response)inresults:print(f"{request.state['color']}:",response)if__name__=="__main__":asyncio.run(main())# red: Passionate and intense, red is a vibrant hue that evokes feelings of love, power, and energy.# green: Green is the vibrant hue of nature, symbolizing growth, freshness, and harmony.# blue: Blue is a cool and calming color that evokes feelings of tranquility and serenity.DesignI tried to keep the design of tokreate as simple as I possibly could. The library is composed of two main classes: aTurnandAction.Turnsare serializable objects that represent a single turn in a conversation.Actionsare objects that use prompts to call an LLMs. Actions generate turns.Turn APIsUnder the hood, turns are Dataclasses objects. Each turn has the following fields:Turn.role: this can be eitheruserorassistant, depending on who generated the turn.Turn.content: the output of the LLM if the turn was generated by an action. Otherwise, this is the prompt from an action.Turn.state: a dictionary that can be used to store any information. Typically this is either variables provided when calling therunmethod of an action, or the output of aParseAction.Turn.meta: a dictionary that stores metadata, typically from calling an LLM api.For example, the following code produces a turn with the following content:fromtokreateimportCallActiongreetings=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="gpt-3.5-turbo")user_turn,assistant_turn=greetings.run(name="Alice")print("User turn:")print('\trole:',user_turn.role)print('\tcontent:',user_turn.content)print('\tstate:',user_turn.state)print('\tmeta:',user_turn.meta)print('\n')print("Assistant turn:")print('\trole:',assistant_turn.role)print('\tcontent:',assistant_turn.content)print('\tstate:',assistant_turn.state)print('\tmeta:',assistant_turn.meta)# User turn:#    role: user#    content: Hello, my name is Alice; what's yours?#    state: {'name': 'Alice'}#    meta: {}# Assistant turn:#   role: assistant#   content: Hello Alice, nice to meet you! I'm ChatGPT. How can I assist you today?#   state: {'name': 'Alice'}#   meta: {'tokens_prompt': 18, 'tokens_completion': 21, 'latency': 0.47, 'messages': [{'role': 'user', 'content': "Hello, my name is Alice; what's yours?"}], 'temperature': 0, 'max_tokens': 300}Note thatstr(turn)returns thecontentof the turn, andrepr(turn)returns '{role}>>> {content}'.Action APIsActions are objects that use prompts to call an LLMs. Actions are run through therun()method, and always return a list of turns (i.e., a history). They can be chained together using>>and<<operators. A chain is itself an action.The library supports two types of actions:CallActionandParseAction. In theory, one could add actions for retrieval, planning, etc. In practice, I don't think those are in scope for this library, and can be added by users.CallActionACallActionis an action that calls an LLMs. It requires two parameters:prompt: the prompt to use when calling the LLMs. Prompts are parsed usingjinja2; each prompt has access to all variables passed by the user when calling therun()method of the action; thestateof the previous turn, and thehistoryof the conversation so far.model: the name of the model to use. This is a string that can be found by runningpython -m tokreate.registry.For example, the following chain of action uses many advanced features of jinja2 templating language:fromtokreateimportCallActionsystem="You are a conversational assistant. You like to talk, but prefer giving short answers."greetings_action=CallAction(prompt="Hello, my name is {{ name }}; what's yours?",model="gpt-3.5-turbo")hobbies_action=CallAction(prompt="What are your hobbies? Mine are {% for hobby in hobbies %}{{ hobby }}{% if not loop.last %}, {% else %}.{% endif %}{% endfor %}",model="gpt-3.5-turbo")color_action=CallAction(prompt="What's your favorite color? Mine is {{ favorite_color }}",model="gpt-3.5-turbo")reflect_prompt="""So far, you told me that:{% for turn in history %}{% if turn.role == "assistant" %}- "{{ turn.content.strip() }}"{% endif %}{% endfor %}Please reflect on what you told me, and what it says about you.""".strip()reflect_action=CallAction(prompt=reflect_prompt,model="gpt-3.5-turbo")history=(greetings_action>>hobbies_action>>color_action>>reflect_action).run(name="Alice",hobbies=["reading","writing","running"],favorite_color="blue")forturninhistory:print(repr(turn))ParseActionThe parse action is used to convert the content of the last turn into a conversation using a function. The output of the conversion is saved in the state of the turn.For example, a parse action can be used to convert the output of an LLMs into a JSON object:importjsonfromtokreateimportParseActionfromtokreate.tokreateimportTurnturn=Turn(role="assistant",content='{"numbers": [4, 9, 2]}')json_parser=ParseAction(parser=json.loads,name="structured_output")*_,parsed_turn=json_parser.step(history=[turn])assertparsed_turn.state["structured_output"]=={"numbers":[4,9,2]}Note hownameis used to specify the key in the state where the output of the parser should be saved. Ifnameis not specified, the output of the parser is save at{function_module}.{function_name}(in this case, it would be atparsed_turn.state["json.loads"]).Supported APIsThis library currently supports OpenAI, Anthropic, and TogetherAI. For a list of all models, runpython -m tokreate.registry. Please open anissue on GitHubto request support for a new API.GPT Json Output (New! 11/06)Tokreate has been updated to keep advantage of the newest GPT-3 API features. For example, using-1106models, we can ensure that the output of the API is a valid JSON object:fromtokreateimportCallActionrandom_action=CallAction(model="gpt-3.5-turbo-1106",prompt="Give me three random numbers as a JSON array",parameters={'response_format':{'type':'json_object'}})*,response=random_action.run()print(response)# {#   "numbers": [5, 23, 17]# }