Package: llm-as-function
llm-as-functionEmbed LLM into your python functionllm-as-functionis a Python library that helps you quickly build functions based on Large Language Models (LLMs). You can useLLMFuncas a decorator for your functions, while also providing type annotations and writing docstrings for your functions.llm-as-functionwill automatically complete the parameter filling by invoking the large model and return formatted output.llm-as-function是一个帮助你快速构建基于LLM的函数的Python库. 你可以使用LLMFunc作为你函数的装饰器, 同时给你的函数进行类型标注和docstring的编写,llm-as-function会自动的通过调用大模型完成参数的填入, 并且返回格式化的输出.Thellm-as-functionalso supports defining code bodies within the LLM function for precise inference control and business logic.llm-as-function还支持在LLM函数中定义代码体, 用于精确的推理控制和业务逻辑.Get Startedpip install llm-as-functionFeaturesBasic usage:fromllm_as_functionimportgpt35_func# gpt35_func using gpt-3.5-turbo as the LLM backendfrompydanticimportBaseModel,Field# Validate and Define output types with PydanticclassResult(BaseModel):emoji:str=Field(description="The output emoji")# Using decorators, LLMFunc will automatically recognize the input and output of your function, as well as the function's docstring.# Here, the function's DocString is your Prompt, so please design it carefully.@gpt35_funcdeffool()->Result:"""You need to randomly output an emoji"""passprint(foo())# {emoji: "😅"}You can also dynamically insert variables into the prompt.@gpt35_funcdeffool2(emotion)->Result:"""You need to randomly output an emoji, the emoji should be {emotion}"""passprint(foo2(emotion="Happy"))# {'emoji': '😊'}You can construct more complex output logic.classReason(BaseModel):where:str=Field(description="Where I can use this emoji?")warning:str=Field(description="Anything I should notice to use this emoji?")classStructuredOutput(BaseModel):emoji:str=Field(description="The emoji")why:str=Field(description="Why you choose this emoji?")more:Reason=Field(description="More information about this emoji")classResult(BaseModel):emoji:StructuredOutput=Field(description="The emoji and its related information")@gpt35_funcdeffool()->Result:"""You need to randomly output an emoji"""passprint(fool())You will get the below output:Final(pack={'emoji':{'emoji':'🎉','why':'I choose this emoji because it represents celebration and excitement.','more':{'where':'I can use this emoji to express joy and happiness in messages or social mediaposts.','warning':'Be mindful of the context in which you use this emoji, as it may not be appropriateforallsituations.'}}},raw_response=None)The most crucial partis that you can insertpythoncode into your function, which will run before the actual LLM execution, so you can accomplish similar tasks:@gpt35_funcdeffool()->Result:"""You need to output an emoji"""print("Logging once")More interestingly, you can invoke other functions within it, other LLM functions, such as calling itself (refer toexamples/3_fibonacci.py):fromllm_as_functionimportgpt35_func,FinalclassResult(BaseModel):value:int=Field(description="The calculated value of the Fibonacci sequence.")@gpt35_funcdeff(x:int)->Result:"""You need to calculate the {x}th term of the Fibonacci sequence, given that you have the values of the two preceding terms, which are {a} and {b}. The method to calculate the {x}th term is by adding the values of the two preceding terms. Please compute the value of the {x}th term."""ifx==1orx==0:# The `Final` is a class in `llm-as-function`, and returning this class indicates that you do not need the large model to process your output. Inside `Final` should be a dictionary, and its format should be the same as the `Result` you defined.returnFinal({"value":x})a=f(x=x-1)b=f(x=x-2)# A normal function return indicates that you have passed 'local variables' to the large model, and the variables you return will be inserted into your prompt.return{"a":a.unpack()["value"],"b":b.unpack()["value"]}print(f(3))# {value: 2}More demos inexamples/DocsLLMFunc# LLMFunc currently support OpenAI provider, Ernie SDK(文心一言)@LLMFunc(model="gpt-3.5-turbo-1106",temperature=0.3,openai_base_url=...,openai_api_key=...)deffool()->Result:...@LLMFunc(model="ernie-bot-4",temperature=0.3)deffool()->Result:...# For your convenience, llm-as-function already instantiated some LLMFuncfromllm_as_functionimportgpt35_func,gpt4_func,ernie_funcx@gpt35_funcdeffool()->Result:...# Parse mode: ["error", "accept_raw"], default "error"# llm-as-function may fail to return the result format, due to LLM doesn't always obey# When the parsing fails, there are two mode you can choose:@LLMFunc(parse_mode="error")deffool()->Result:...result=fool()# When the parsing fails, fool will raise an error@LLMFunc(parse_mode="accept_raw")deffool()->Result:...result=fool()# When the parsing fails, fool will not raise an error but return the raw response of LLM, refer to the `Final` classFinal# The return value of any llm function is a `Final` classresult:Final=fool()ifresult.ok():format_response=result.unpack()# the response will be a formated dictelse:raw_response=result.unpack()# the response will be the raw string result from LLMFQAThe formatting of the return fromllm-as-functiondepends on the capabilities of the model you are using. Sometimes, larger models may not be able to return a parsable JSON format, which can lead to an Error or return the raw response if you set theparse_mode="accept_raw".当llm-as-function使用的是ernie-bot-4, 其API的访问对于rate limit限制的比较狠, 如果你遇到如下的Errorerniebot.errors.APIError: Max retry is reached代表你遇到rate limit限制了, 考虑进行换模型或者在每次使用function后sleep一段时间