Package: llm-wrapper-allegro
llm-wrapperAboutllm-wrapper is a versatile and powerful library designed to streamline the process of querying Large Language Models
(LLMs) 🤖💬Developed by the Allegro engineers, llm-wrapper is based on popular libraries like transformers, pydantic, and langchain. It takes care
of the boring boiler-plate code you write around your LLM applications, quickly enabling you to prototype ideas, and eventually helping you to scale up
for production use-cases!Among the llm-wrapper most notable features, you will find:😊 Simple and User-Friendly Interface: The module offers an intuitive and easy-to-use interface, making it straightforward to work with the model.🔀 Asynchronous Querying: Requests to the model are processed asynchronously by default, ensuring efficient and non-blocking interactions.🔄 Automatic Retrying Mechanism: The module includes an automatic retrying mechanism, which helps handle transient errors and ensures that queries to the model are robust.🛠️ Error Handling and Management: Errors that may occur during interactions with the model are handled and managed gracefully, providing informative error messages and potential recovery options.⚙️ Output Parsing: The module simplifies the process of defining the model's output format as well as parsing and working with it, allowing you to easily extract the information you need.DocumentationFull documentation available atllm-wrapper.allegro.techGet familiar with llm-wrapper 🚀:introductory jupyter notebookQuickstartInstallation 🚧Install the package via pip:pip install llm-wrapperBasic Usage ⭐Configure endpoint credentials and start querying the model with any prompt:fromllm_wrapper.modelsimportAzureOpenAIModelfromllm_wrapper.domain.configurationimportAzureOpenAIConfigurationconfiguration=AzureOpenAIConfiguration(api_key="your-secret-api-key",base_url="https://endpoint.openai.azure.com/",api_version="2023-03-15-preview",deployment="gpt-35-turbo",model_name="gpt-3.5-turbo")gpt_model=AzureOpenAIModel(config=configuration)gpt_response=gpt_model.generate(prompt="Plan me a 3-day holiday trip to Italy")You can pass also a system prompt:gpt_response=gpt_model.generate(system_prompt="You are an AI assistant acting as a trip planner",prompt="Plan me a 3-day holiday trip to Italy")Advanced Usage 🔥Batch Querying and Symbolic VariablesIf you want to generate responses for a batch of examples, you can achieve this by preparing a prompt with symbolic
variables and providing input data that will be injected into the prompt. Symbolic variables can be more than one.positive_review_0="Very good coffee, lightly roasted, with good aroma and taste. The taste of sourness is barely noticeable (which is good because I don't like sour coffees). After grinding, the aroma spreads throughout the room. I recommend it to all those who do not like strongly roasted and pitch-black coffees. A very good solution is to close the package with string, which allows you to preserve the aroma and freshness."positive_review_1="Delicious coffee!! Delicate, just the way I like it, and the smell after opening is amazing. It smells freshly roasted. Faithful to Lavazza coffee for years, I decided to look for other flavors. Based on the reviews, I blindly bought it and it was a 10-shot, it outperformed Lavazze in taste. For me the best."negative_review="Marketing is doing its job and I was tempted too, but this coffee is nothing above the level of coffees from the supermarket. And the method of brewing or grinding does not help here. The coffee is simply weak - both in terms of strength and taste. I do not recommend."prompt="You'll be provided with a review of a coffe. Decide if the review is positive or negative. Review:{review}"input_data=[InputData(input_mappings={"review":positive_review_0},id="0"),InputData(input_mappings={"review":positive_review_1},id="1"),InputData(input_mappings={"review":negative_review},id="2")]responses=model.generate(prompt=prompt,input_data=input_data)# >>> {f"review_id={response.input_data.id}": response.response for response in responses}# {#     'review_id=0': 'The review is positive.',#     'review_id=1': 'The review is positive.',#     'review_id=2': 'The review is negative.'# }Forcing Structured Output FormatThrough pydantic integration, in llm-wrapper you can pass an output dataclass and force the LLM to provide
you the response in a structured way.frompydanticimportBaseModel,FieldclassReviewOutputDataModel(BaseModel):summary:str=Field(description="Summary of a product description")should_buy:bool=Field(description="Recommendation whether I should buy the product or not")brand_name:str=Field(description="Brand of the coffee")aroma:str=Field(description="Description of the coffee aroma")cons:list[str]=Field(description="List of cons of the coffee")review="Marketing is doing its job and I was tempted too, but this Blue Orca coffee is nothing above the level of coffees from the supermarket. And the method of brewing or grinding does not help here. The coffee is simply weak - both in terms of strength and taste. I do not recommend."prompt="Summarize review of the coffee. Review:{review}"input_data=[InputData(input_mappings={"review":review},id="0")]responses=model.generate(prompt=prompt,input_data=input_data,output_data_model_class=ReviewOutputDataModel)response=responses[0].response# >>> type(response)# ReviewOutputDataModel## >>> response.should_buy# False## >>> response.brand_name# "Blue Orca"## >>> response.aroma# "Not mentioned in the review"## >>> response.cons# ['Weak in terms of strength', 'Weak in terms of taste']Local Development 🛠️Installation from the sourceWe assume that you have python3.10.*installed on your machine.
You can set it up usingpyenv(How to install pyenv on MacOS). To install llm-wrapper env locally:Activate your pyenv;Install Poetry via:makeinstall-poetryInstall llm-wrapper dependencies with the command:makeinstall-envNow you can use this venv for development. You can activate it in your shell by running:makeactivate-env# or simply, poetry shellTestsIn order to execute tests, run:maketestsUpdating the documentationRunmkdocs serveto serve a local instance of the documentation.Modify the content ofdocsdirectory to update the documentation. The updated content will be deployed
via the github action.github/workflows/docs.ymlMake a new releaseWhen a new version of llm-wrapper is ready to be released, do the following operations:Merge to masterthe dev branch in which the new version has been specified:In this branch,versionunder[tool.poetry]section inpyproject.tomlshould be updated, e.g0.1.0;Update the CHANGELOG, specifying the new release.Tag the new masterwith the name of the newest version:e.gv0.1.0.Publish package to PyPI:Go toActions→Manual Publish To PyPI;Select "master" as branch and clickRun workflow;If successful, you will find the package under # TODO: open-source.Make a GitHub release:Go toReleases→Draft a new release;Select the recently created tag inChoose a tagwindow;Copy/paste all the content present in the CHANGELOG under the version you are about to release;Uploadllm_wrapper-<NEW-VERSION>.whlandllm_wrapper-<NEW-VERSION>.tar.gzas assets;ClickPublish release.