Package: llfn
LLFn (read: 🐘 Elephant)is a light-weight framework for creating applications using LLMs.Build anything from a simple text-summarizer to complex AI agents with one common primitive:function.Core Concept·Installation·Examples·Report Bug·Request FeatureFeaturesLLFnhas only one goal: making it 100x easier to experiment and ship AI applications using the LLMs with minimum boilerplate. Even if you're new to LLMs, you can still get started and be productive in minutes, instead of spending hours learning how to use the framework.🔋 Functionify: turning anypromptinto a callable function.📤 Flexible Output: defining LLM result with Python andpydantictypes.🧱 Infinitely Composable: any function can call any other function in any order.🛒 Use Any LLM: leveragingLangChain's LLM interface, drop in your favorite LLM with 100% compatibility.🪶 Light-Weight: small core framework of LLFn making it extremely easy to understand and extend.We draw a lot of inspiration from bothFastAPIandLangChain, so if you're familiar with either of them you'd feel right at home with LLFn.Core Concept:Function is All You Need⭐️ Defining Your FunctionThe primary goal of LLFn is to encapsulate everything you do with LLMs into a function. Each function comprises of 3 components:input,prompt, andoutput. Here's how you can create a simple text translator using LLFn.fromllfnimportLLFnfunction_prompt=LLFn()@function_promptdeftranslate(text:str,output_language:str)->str:returnf"Translate the following text into{output_language}language:{text}"🥪 Binding It with LLMTo execute this you have tobindthe function to an LLM. You can bind anyLangChain's supported language modelsto LLFn functions.fromlangchain.chat_modelsimportChatOpenAI# Setup LLM and bind it to the functionllm=ChatOpenAI(temperature=0.7,openai_api_key=API_KEY)translate.bind(llm)If you don't want to repeat yourself binding all the functions individually, you can bind directly tofunction_promptas well.# Alternatively: bind the llm to all functionsfunction_prompt.bind(llm)🍺 Calling the FunctionNow you can call your function like you would for any Python function.translate("Hello welcome. How are you?","Thai")# สวัสดี ยินดีต้อนรับ สบายดีไหม?The beauty of this construct is that you're able to infinitely compose your applications. LLFn does not make any assumption on how you'd chain up your application logic. The only requirement for afunction_promptis that it returns a prompt string at the end.And ... that's it! That's just about everything you need to learn about LLFn to start building AI apps with it.👆 Click to see the full codeimportosfromllfnimportLLFnfromlangchain.chat_modelsimportChatOpenAIllm=ChatOpenAI(temperature=0.7,model=os.getenv("OPENAI_MODEL"),openai_api_key=os.getenv("OPENAI_API_KEY"),)function_prompt=LLFn()function_prompt.bind(llm)@function_promptdeftranslate(text:str,output_language:str)->str:returnf"Translate the following text into{output_language}language:{text}"# Call the functionprint(translate("Hello welcome. How are you?","Thai"))InstallationLLFn is available onPyPi. You can install LLFn using your favorite Python package management software.$pipinstallllfn# If you use pip$poetryaddllfn# If you use poetryAdvanced FeaturesWhile LLFn is simple and light-weight by design, it does pack some punches above its weight that helps you write AI applications more intuitively.📐 Structured Function OutputYou can create a function to automatically format the response into a desiredpydanticobject. To do this, you simply declare the output of your function inpydantic, and let the LLFn do the heavy lifting for you.frompydanticimportBaseModel,FieldclassAgentLogicOutput(BaseModel):tasks:List[str]=Field(...,description="list of tasks to accomplish the given goal")reasoning:str=Field(...,description="why you choose to do these tasks")@function_promptdefagent_think(goal:str)->AgentLogicOutput:returnf"You're an agent planning 5 tasks to accomplish the following goal:{goal}"agent_think("Creating online business")# this returns AgentLogicOutput objectThis allows for powerful composability like tools, agents, etc. without sacrificing the simplicity and verbosity of our program. Under the hood LLFn injects the output format into your prompt and automatically parse the LLM result into the specified format.🗝️ Binding and Overriding LLMsLLFn let you use both LangChain'sLLMsandChat Modelsto power your functions. There are 2 ways toassignit.Method 1: Binding LLM to All Your FunctionsThis is the most convenient way to just get your function to work.fromllfnimportLLFnfromlangchain.chat_modelsimportChatOpenAIllm=ChatOpenAI(temperature=0.7,openai_api_key=API_KEY)function_prompt=LLFn()function_prompt.bind(llm)@function_promptdefyour_function(...):...Method 2: Binding/Overriding LLM to A Specific FunctionIf you want to use different LLMs for different tasks, you can do so by bindingllmto your function individually.fromlangchain.llmsimportLlamaCppllm2=LlamaCpp(...)your_function.bind(llm2)# This will override the LLM for this function🏄‍♂️ Callbacks and StreamingCallbacksWith LLFn, it's trivial to define callbacks in between execution. Because everything is a function, you canprintor call any 3rd party APIs anywhere you want.For example, if you want to print results of executions in between a series of complex AI agent it would look like this....# Assuming that other functions are already defined with function_prompt@function_promptdefagent_plan_and_execute(goal:str)->str:tasks=agent_think(goal)print(tasks)# Debugresults=agent_execute_tasks(tasks)print(results)# Debugevaluation=agent_evaluate_perf(goal,results)print(evaluation)# DebugYour function can also take a callback function as a parameter. LLFn does not dictate how you should do it. Anything that works for you would do!Streaming LLM OutputBecause LLFn leverages LangChain's LLMs, you can useLangChain Streamingdirectly.fromlangchain.callbacks.streaming_stdoutimportStreamingStdOutCallbackHandlerllm=ChatOpenAI(streaming=True,callbacks=[StreamingStdOutCallbackHandler()],...)function_prompt.bind(llm)Examples⭐️ Single-Prompt ExamplesTaskShowcaseMachine TranslationUsing LangChain Chat ModelMachine TranslationUsing LangChain LLM ModelAgent Task PlannerDefining Structured Output🌈 Multi-Prompt Complex ExamplesTaskShowcaseBabyAGIImplementing BabyAGI with LLFnSponsors and ContributorsWe currently don't take any monetary donations! However, every issue filed and PR are extremely important to us. Here is the roster of contributors and supporters of the project.Contribution and FeedbackContributions, feedback, and suggestions are highly encouraged and appreciated! You can contribute to LLFn in the following ways:🔀 Fork the repository and make modifications in your own branch. Then, submit a pull request (PR) with your changes.⬇️ Submit issues (GitHub Issues) to report bugs, suggest new features, or ask questions.CitationIf you wish to cite LLFn in your research, we encourage the use ofCITATION.cffprovided for appropriate citation formatting. For more details on the citation file format, please visit theCitation File Format website.LicenseLLFn is an open-source software under theMIT license. This license allows use, modification, and distribution of the software. For complete details, please see theLICENSEfile in this repository.AcknowledgementsWe would like to express our gratitude to the following projects and communities for their inspiring work and valuable contributions:LangChainPydanticFastAPIOpenAI