Package: python-aiconfig
Full documentation:aiconfig.lastmileai.devOverviewAIConfig saves prompts, models and model parameters as source control friendly configs. This allows you to iterate on prompts and model parametersseparately from your application code.Prompts as configs: astandardized JSON formatto store generative AI model settings, prompt inputs/outputs, and flexible metadata.Model-agnostic SDK: Python & Node SDKs to useaiconfigin your application code. AIConfig is designed to bemodel-agnosticandmulti-modal, so you can extend it to work with any generative AI model, including text, image and audio.AI Workbook editor: Anotebook-like playgroundto editaiconfigfiles visually, run prompts, tweak models and model settings, and chain things together.What problem it solvesToday, application code is tightly coupled with the gen AI settings for the application -- prompts, parameters, and model-specific logic is all jumbled in with app code.results in increased complexitymakes it hard to iterate on the prompts or try different models easilymakes it hard to evaluate prompt/model performanceAIConfig helps unwind complexity by separating prompts, model parameters, and model-specific logic from your application.simplifies application code -- simply callconfig.run()open theaiconfigin a playground to iterate quicklyversion control and evaluate theaiconfig- it's the AI artifact for your application.QuicknavGetting StartedCreate an AIConfigRun a promptPass data into promptsPrompt ChainsCallbacks and monitoringSDK CheatsheetCookbooks and guidesCLI ChatbotRAG with ChromaDBRAG with MongoDBPrompt routingOpenAI function callingChain of VerificationSupported modelsLLaMA2 exampleHugging Face (Mistral-7B) examplePaLMExtensibilityContributingRoadmapFAQFeaturesSource-control friendlyaiconfigformatto save prompts and model settings, which you can use for evaluation, reproducibility and simplifying your application code.Multi-modal and model agnostic. Use with any model, and serialize/deserialize data with the sameaiconfigformat.Prompt chaining and parameterizationwith{{handlebars}}templating syntax, allowing you to pass dynamic data into prompts (as well as between prompts).Streamingsupported out of the box, allowing you to get playground-like streaming wherever you useaiconfig.Notebook editor.AI Workbooks editorto visually create youraiconfig, and use the SDK to connect it to your application code.InstallInstall with your favorite package manager for Node or Python.Node.jsnpmoryarnnpminstallaiconfigyarnaddaiconfigPythonpiporpoetrypipinstallpython-aiconfigpoetryaddpython-aiconfigDetailed installation instructions.Getting StartedWe cover Python instructions here, for Node.js please see thedetailed Getting Started guideIn this quickstart, you will create a customizable NYC travel itinerary usingaiconfig.This AIConfig contains a prompt chain to get a list of travel activities from an LLM and then generate an itinerary in an order specified by the user.Link to tutorial code:herehttps://github.com/lastmile-ai/aiconfig/assets/25641935/d3d41ad2-ab66-4eb6-9deb-012ca283ff81Downloadtravel.aiconfig.jsonNote: Don't worry if you don't understand all the pieces of this yet, we'll go over it step by step.{"name":"NYC Trip Planner","description":"Intrepid explorer with ChatGPT and AIConfig","schema_version":"latest","metadata":{"models":{"gpt-3.5-turbo":{"model":"gpt-3.5-turbo","top_p":1,"temperature":1},"gpt-4":{"model":"gpt-4","max_tokens":3000,"system_prompt":"You are an expert travel coordinator with exquisite taste."}},"default_model":"gpt-3.5-turbo"},"prompts":[{"name":"get_activities","input":"Tell me 10 fun attractions to do in NYC."},{"name":"gen_itinerary","input":"Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.","metadata":{"model":"gpt-4","parameters":{"order_by":"geographic location"}}}]}Run theget_activitiesprompt.Note: Make sure to specify the API keys (such asOPENAI_API_KEY) in your environment before proceeding.exportOPENAI_API_KEY=my_keyYou don't need to worry about how to run inference for the model; it's all handled by AIConfig. The prompt runs with gpt-3.5-turbo since that is thedefault_modelfor this AIConfig.importasynciofromaiconfigimportAIConfigRuntime,InferenceOptionsasyncdefmain():# Load the aiconfigconfig=AIConfigRuntime.load('travel.aiconfig.json')# Run a single prompt (with streaming)inference_options=InferenceOptions(stream=True)awaitconfig.run("get_activities",options=inference_options)asyncio.run(main())Run thegen_itineraryprompt.This prompt depends on the output ofget_activities. It also takes in parameters (user input) to determine the customized itinerary.Let's take a closer look:gen_itineraryprompt:"Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}."prompt metadata:{"metadata":{"model":"gpt-4","parameters":{"order_by":"geographic location"}}}Observe the following:The prompt depends on the output of theget_activitiesprompt.It also depends on anorder_byparameter (using {{handlebars}} syntax)It usesgpt-4, whereas theget_activitiesprompt it depends on usesgpt-3.5-turbo.Effectively, this is a prompt chain betweengen_itineraryandget_activitiesprompts,as well asas a model chain betweengpt-3.5-turboandgpt-4.Let's run this with AIConfig:Replaceconfig.runabove with this:awaitconfig.run("gen_itinerary",params={"order_by":"duration"},options=inference_options,run_with_dependencies=True)Notice how simple the syntax is to perform a fairly complex task - running 2 different prompts across 2 different models and chaining one's output as part of the input of another.The code will just runget_activities, then pipe its output as an input togen_itinerary, and finally rungen_itinerary.Save the AIConfigLet's save the AIConfig back to disk, and serialize the outputs from the latest inference run as well:# Save the aiconfig to disk. and serialize outputs from the model runconfig.save('updated.aiconfig.json',include_outputs=True)Editaiconfigin a notebook editorWe can iterate on anaiconfigusing a notebook-like editor called anAI Workbook. Now that we have anaiconfigfile artifact that encapsulates the generative AI part of our application, we can iterate on it separately from the application code that uses it.Go tohttps://lastmileai.dev.Go to Workbooks page:https://lastmileai.dev/workbooksClick dropdown from '+ New Workbook' and select 'Create from AIConfig'Uploadtravel.aiconfig.jsonhttps://github.com/lastmile-ai/aiconfig/assets/81494782/5d901493-bbda-4f8e-93c7-dd9a91bf242eTry out the workbook playground here:NYC Travel WorkbookWe are working on a local editor that you can run yourself. For now, please use the hosted version onhttps://lastmileai.dev.Additional GuidesThere is a lot you can do withaiconfig. We have several other tutorials to help get you started:Create an AIConfig from scratchRun a promptPass data into promptsPrompt chainsCallbacks and monitoringHere are some example uses:CLI ChatbotRAG with AIConfigPrompt routingOpenAI function callingChain of thoughtOpenAI Introspection APIIf you are already using OpenAI completion API's in your application, you can get started very quickly to start saving the messages in anaiconfig.Usage: see openai_wrapper.ipynb.Now you can continue usingopenaicompletion API as normal. When you want to save the config, just callnew_config.save()and all your openai completion calls will get serialized to disk.Detailed guide hereSupported ModelsAIConfig supports the following model models out of the box:OpenAI chat models (GPT-3, GPT-3.5, GPT-4)LLaMA2 (running locally)Google PaLM models (PaLM chat)Hugging Face text generation models (e.g. Mistral-7B)ExamplesOpenAILLaMA exampleHugging Face (Mistral-7B) examplePaLMIf you need to use a model that isn't provided out of the box, you can implement aModelParserfor it (seeExtending AIConfig).We welcomecontributionsAIConfig SchemaAIConfig specificationAIConfig SDKRead theUsage Guidefor more details.The AIConfig SDK supports CRUD operations for prompts, models, parameters and metadata. Here are some common examples.The root interface is theAIConfigRuntimeobject. That is the entrypoint for interacting with an AIConfig programmatically.Let's go over a few key CRUD operations to give a glimpse.AIConfigcreateconfig=AIConfigRuntime.create("aiconfig name","description")Promptresolveresolvedeserializes an existingPromptinto the data object that its model expects.config.resolve("prompt_name",params)paramsare overrides you can specify to resolve any{{handlebars}}templates in the prompt. See thegen_itineraryprompt in the Getting Started example.Promptserializeserializeis the inverse ofresolve-- it serializes the data object that a model understands into aPromptobject that can be serialized into theaiconfigformat.config.serialize("model_name",data,"prompt_name")Promptrunrunis used to run inference for the specifiedPrompt.config.run("prompt_name",params)run_with_dependenciesThis is a variant ofrun-- this re-runs all prompt dependencies.
For example, intravel.aiconfig.json, thegen_itineraryprompt references the output of theget_activitiesprompt using{{get_activities.output}}.Running this function will first executeget_activities, and use its output to resolve thegen_itineraryprompt before executing it.
This is transitive, so it computes the Directed Acyclic Graph of dependencies to execute. Complex relationships can be modeled this way.config.run_with_dependencies("gen_itinerary")Updating metadata and parametersUse theget/set_metadataandget/set_parametermethods to interact with metadata and parameters (set_parameteris just syntactic sugar to update"metadata.parameters")config.set_metadata("key",data,"prompt_name")Note: if"prompt_name"is specified, the metadata is updated specifically for that prompt. Otherwise, the global metadata is updated.Register newModelParserUse theAIConfigRuntime.register_model_parserif you want to use a differentModelParser, or configure AIConfig to work with an additional model.AIConfig uses the model name string to retrieve the rightModelParserfor a given Prompt (seeAIConfigRuntime.get_model_parser), so you can register a different ModelParser for the same ID to override whichModelParserhandles a Prompt.For example, suppose I want to useMyOpenAIModelParserto handlegpt-4prompts. I can do the following at the start of my application:AIConfigRuntime.register_model_parser(myModelParserInstance,["gpt-4"])Callback eventsUse callback events to trace and monitor what's going on -- helpful for debugging and observability.fromaiconfigimportAIConfigRuntime,CallbackEvent,CallbackManagerconfig=AIConfigRuntime.load('aiconfig.json')asyncdefmy_custom_callback(event:CallbackEvent)->None:print(f"Event triggered:{event.name}",event)callback_manager=CallbackManager([my_custom_callback])config.set_callback_manager(callback_manager)awaitconfig.run("prompt_name")Read morehereExtensibilityAIConfig is designed to be customized and extended for your use-case. TheExtensibilityguide goes into more detail.Currently, there are 3 core ways to extend AIConfig:Supporting other models- define a ModelParser extensionCallback event handlers- tracing and monitoringCustom metadata- save custom fields inaiconfigContributing toaiconfigThis is our first open-source project and we'd love your help.See ourcontributing guidelines-- we would especially love help adding support for additional models that the community wants.CookbooksWe provide several guides to demonstrate the power ofaiconfig.See thecookbooksfolder for examples to clone.ChatbotWizard GPT- speak to a wizard on your CLICLI-mate- help you make code-mods interactively on your codebase.Retrieval Augmented Generated (RAG)RAG with AIConfigAt its core, RAG is about passing data into prompts. Read how topass datawith AIConfig.Function callingOpenAI function callingPrompt routingPrompt routingChain of ThoughtA variant of chain-of-thought is Chain of Verification, used to help reduce hallucinations. Check out the aiconfig cookbook for CoVe:Chain of VerificationUsing local LLaMA2 withaiconfigLLaMA exampleHugging Face text generationHugging Face (Mistral-7B) exampleGoogle PaLMPaLMRoadmapThis project is under active development.If you'd like to help, please see thecontributing guidelines.Please create issues for additional capabilities you'd like to see.Here's what's already on our roadmap:Evaluation interfaces: allowaiconfigartifacts to be evaluated with user-defined eval functions.We are also considering integrating with existing evaluation frameworks.Local editor foraiconfig: enable you to interact with aiconfigs more intuitively.OpenAI Assistants API supportMulti-modal ModelParsers:GPT4-V supportDALLE-3WhisperHuggingFace image generationFAQsHow should I edit anaiconfigfile?Editing a configshould be done either programmatically via SDK or via the UI (workbooks):Programmaticediting.Edit with a workbookeditor: this is similar to editing an ipynb file as a notebook (most people never touch the json ipynb directly)You should only edit theaiconfigby hand for minor modifications, like tweaking a prompt string or updating some metadata.Does this support custom endpoints?Out of the box, AIConfig already supports all OpenAI GPT* models, Google’s PaLM model and any “textgeneration” model on Hugging Face (like Mistral). SeeSupported Modelsfor more details.Additionally, you can installaiconfigextensionsfor additional models (see question below).Is OpenAI function calling supported?Yes.This examplegoes through how to do it.We are also working on adding support for the Assistants API.How can I use aiconfig with my own model endpoint?Model support is implemented as “ModelParser”s in the AIConfig SDK, and the idea is that anyone, including you, can define a ModelParser (and even publish it as an extension package).All that’s needed to use a model with AIConfig is a ModelParser that knowshow to serialize data from a model into the aiconfig formathow to deserialize data from an aiconfig into the type the model expectshow to run inference for model.For more details, seeExtensibility.When should I store outputs in anaiconfig?TheAIConfigRuntimeobject is used to interact with an aiconfig programmatically (seeSDK usage guide). As you run prompts, this object keeps track of the outputs returned from the model.You can choose to serialize these outputs back into theaiconfigby using theconfig.save(include_outputs=True)API. This can be useful for preserving context -- think of it like session state.For example, you can use aiconfig to create a chatbot, and use the same format to save the chat history so it can be resumed for the next session.You can also choose to save outputs to adifferentfile than the original config --config.save("history.aiconfig.json", include_outputs=True).Why should I useaiconfiginstead of things likeconfigurator?It helps to have astandardized formatspecifically for storing generative AI prompts, inference results, model parameters and arbitrary metadata, as opposed to a general-purpose configuration schema.With that standardization, you just need a layer that knows how to serialize/deserialize from that format into whatever the inference endpoints require.This looks similar toipynbfor Jupyter notebooksWe believe that notebooks are a perfect iteration environment for generative AI -- they are flexible, multi-modal, and collaborative.The multi-modality and flexibility offered by notebooks andipynboffers a good interaction model for generative AI. Theaiconfigfile format is extensible likeipynb, and AI Workbook editor allows rapid iteration in a notebook-like IDE.AI Workbooks are to AIConfig what Jupyter notebooks are toipynbThere are 2 areas where we are going beyond what notebooks offer:aiconfigis moresource-control friendlythanipynb.ipynbstores binary data (images, etc.) by encoding it in the file, whileaiconfigrecommends using file URI references instead.aiconfigcan be imported andconnected to application codeusing the AIConfig SDK.