Package: turbo-chat
turbo-chatIdiomatic way to build chatgpt apps using async generators in pythonAboutTheChatGPT APIuses a new input format calledchatml. In openai'spython client, the format is used something like this:messages=[{"role":"system","content":"Greet the user!"},{"role":"user","content":"Hello world!"},]The idea here is to incrementally build the messages using an async generator and then use that to generate completions.Async generatorsare incredibly versatile and simple abstraction for doing this kind of stuff. They can also be composed together very easily.# Equivalent turbo-chat generatorasyncdefexample():yieldSystem(content="Greet the user!")yieldUser(content="Hello World!")# To run generation, just yield Generate(),# the lib will take care of correctly running the app, and# return the value back here.output=yieldGenerate()print(output.content)See more detailed example below.Installationpipinstallturbo-chatExamplefromtypingimportAsyncGenerator,Unionfromturbo_chatimport(turbo,System,User,Assistant,GetInput,Generate,run,)# Get userasyncdefget_user(id):return{"zodiac":"pisces"}# Set user zodiac mixin# Notice that no `@turbo()` decorator used hereasyncdefset_user_zodiac(user_id:int):user_data:dict=awaitget_user(user_id)zodiac:str=user_data["zodiac"]yieldUser(content=f"My zodiac sign is{zodiac}")# Horoscope app@turbo(temperature=0.0)asyncdefhoroscope(user_id:int):yieldSystem(content="You are a fortune teller")# Yield from mixinasyncforoutputinset_user_zodiac(user_id):yieldoutput# Prompt runner to ask for user inputinput=yieldGetInput(message="What do you want to know?")# Yield the inputyieldUser(content=input)# Generate (overriding the temperature)value=yieldGenerate(temperature=0.9)# Let's run thisapp:AsyncGenerator[Union[Assistant,GetInput],str]=horoscope({"user_id":1})_input=Nonewhilenot(result:=await(app.run(_input)).done:ifresult.needs_input:# Prompt user with the input message_input=input(result.content)continueprint(result.content)# Output# >>> What do you want to know? Tell me my fortune# >>> As an AI language model, I cannot predict the future or provide supernatural fortune-telling. However, I can offer guidance and advice based on your current situation and past experiences. Is there anything specific you would like me to help you with?#Custom memoryYou can also customize how the messages are persisted in-between the executions.fromturbo_chatimportturbo,BaseMemoryclassRedisMemory(BaseMemory):"""Implement BaseMemory methods here"""asyncdefsetup(self,**kwargs)->None:...asyncdefappend(self,item)->None:...asyncdefclear(self)->None:...# Now use the memory in a turbo_chat app@turbo(memory_class=RedisMemory)asyncdefapp():...Get access to memory object directly (just declare an additional param)@turbo()asyncdefapp(some_param:Any,memory:BaseMemory):messages=awaitmemory.get()...Generate a response to use internally but don't yield downstream@turbo()asyncdefexample():yieldSystem(content="You are a good guy named John")yieldUser(content="What is your name?")result=yieldGenerate(forward=False)yieldUser(content="How are you doing?")result=yieldGenerate()b=example()results=[outputasyncforoutputinb]assertlen(results)==1Add a simple in-memory cacheYou can also subclass theBaseCacheclass to create a custom cache.cache=SimpleCache()@turbo(cache=cache)asyncdefexample():yieldSystem(content="You are a good guy named John")yieldUser(content="What is your name?")result=yieldGenerate()b=example()results=[outputasyncforoutputinb]assertlen(cache.cache)==1Latest Changesversion: 0.3.11. PR#62by@creatorrr.fix: Fix truncation. PR#61by@creatorrr.version: 0.3.10. PR#60by@creatorrr.version: 0.3.9. PR#59by@creatorrr.x/fix cache args. PR#58by@creatorrr.version: 0.3.7. PR#57by@creatorrr.f/support multi choice. PR#56by@creatorrr.feat: Support multiple choices selector; n > 1. PR#55by@creatorrr.feat: Support positional arguments for running apps. PR#54by@creatorrr.feat: Make get_encoding faster. PR#53by@creatorrr.feat: Add ttl support to redis_cache. PR#52by@creatorrr.feat: Support for parsing completions. PR#51by@creatorrr.version: 0.3.6. PR#50by@creatorrr.feat: Add RedisCache implementation. PR#49by@creatorrr.fix: Fix json array. PR#48by@creatorrr.version: 0.3.5. PR#47by@creatorrr.fix: Fix how function signature and docstring was being parsed. PR#46by@creatorrr.version: 0.3.4. PR#45by@creatorrr.feat: Add @completion decorator. PR#44by@creatorrr.version: 0.3.3. PR#43by@creatorrr.feat: Memory expects memory_args, Assistant no longer yields automatically. PR#42by@creatorrr.v/0.3.2. PR#41by@creatorrr.version: 0.3.1. PR#40by@creatorrr.fix: Fix scratchpad parsing. PR#39by@creatorrr.version: 0.3.0. PR#38by@creatorrr.x/more toolbot fixes. PR#37by@creatorrr.version: 0.2.13. PR#36by@creatorrr.v/0.2.12. PR#35by@creatorrr.fix: toolbot additional_info parameter. PR#34by@creatorrr.version: 0.2.11. PR#33by@creatorrr.f/json tool bot. PR#32by@creatorrr.version: 0.2.10. PR#31by@creatorrr.feat: Add sticky messages. PR#30by@creatorrr.version: 0.2.9. PR#29by@creatorrr.feat: Add .init method to TurboGenWrapper. PR#28by@creatorrr.doc: Generate FAQ using autodoc. PR#27by@creatorrr.version: 0.2.8. PR#26by@creatorrr.feat: Add .run() method to the TurboGen object. PR#25by@creatorrr.version: 0.2.7. PR#24by@creatorrr.feat: self-ask bot. PR#23by@creatorrr.version: 0.2.6. PR#22by@creatorrr.feat: Add summary memory. PR#21by@creatorrr.version: 0.2.5. PR#20by@creatorrr.version: 0.2.4. PR#19by@creatorrr.refactor: Move trucation logic to a separate memory class. PR#18by@creatorrr.version: 0.2.3. PR#17by@creatorrr.f/memory improvements. PR#16by@creatorrr.version: 0.2.2. PR#15by@creatorrr.f/tool bot. PR#14by@creatorrr.v/0.2.1. PR#13by@creatorrr.feat: Add count_tokens. PR#12by@creatorrr.Update README.md. PR#11by@creatorrr.