M ini C hain: A Small Library for Coding with Large Language Models (2023)

Abstract: Programming augmented by large language models (LLMs) opens up many new application areas, but also requires care. LLMs are accurate enough, on average, to replace core functionality, yet make basic mistakes that demonstrate a lack of robustness. An ecosystem of prompting tools, from intelligent agents to new programming languages, have emerged with different solutions for patching LLMs with other tools. In this work, we introduce MiniChain, an opinionated tool for LLM augmented programming, with the design goals of ease-of-use of prototyping, transparency through automatic visualization, and a minimalistic approach to advanced features. The MiniChain library provides core primitives for coding LLM calls, separating out prompt templates, and capturing program structure. The library includes demo implementations of the main applications papers in the area, including chat-bots, code generation, retrieval-based question answering, and complex information extraction. The library is open-source and available at https://github.com/srush/MiniChain, with code demos available at https://srush-minichain.hf.space/, and video demo at https://www.youtube.com/watch?v=VszZ1VnO7sk.

## introduction large language models (llms) @xcite are a transformative technology that make it possible to develop novel ai applications. out of the box they perform extremely well across many different domains including code generation, question answering and decomposition, fact retrieval, information extraction, and dialogue to name a few, as well as entirely novel task domains. however, while demonstrating these novel behaviors, they also struggle in basic areas such as mathematical reasoning @xcite , code execution @xcite , specific document lookup @xcite , and handling long contexts @xcite . the gap between the novel general-purpose abilities and low-level deficiencies in known areas, has motivated significant research into multi-stage systems, colloquially chains, that combine the use of llms with more basic computation blocks and calls to other classical tools. this intermediary software ecosystem describes compositional structure for how the llm interacts with the scaffolding around it. despite the agreed upon problem, there are many different approaches being pursued simultaneously in the open-source community. systems like auto-gpt (autogpt) utilize a fully autonomous agent to direct the choice of supplementary tooling. other systems like lmql @xcite propose a new query language that is able to fully guide and constrict the llm. in between, the popular langchain @xcite software provides a full-service toolkit for working with many of the different paradigms, from agents to vectordatabases to chat bots with memory. the diversity of these systems indicates a broad and open challenge in designing tools that best facilitate programmer interaction with llms. in this white paper, we propose a system with a different goal along this design space. minichain, is an implementation of the prompt chaining paradigm that can support widely used chaining patterns while remaining significantly simpler than comparable libraries. minichain is designed with three goals: a) ease-of-use, it should be indistinguishable from standard python code, b) transparency, it should be trivial for the user to introspect and follow all ai calls, c) minimal, it should not implement features that can be done easily with code. the library itself and the underlying visualization tool are written in python and follow standard coding conventions. the library aims to be understandable by researchers and is contained in one file. however, it is also meant to be complete, and be possible to implement contemporary research on the topic. to demonstrate this, the whitepaper comes with an implementation of recent prompting research at https://srush-minichain.hf. space/ . in addition there is a video demo describing its use at https://www.youtube.com/watch? @xmath0. ## related work programming with interleaved llm calls is a very recent phenomenon, and so there are relatively comparable systems. of the related systems, many exist as open-source libraries or as demo code, and it is difficult to categorize their evolving features. roughly, prompt chaining systems can be divided into five groups: toolkits for calling llms and managing state. of these the most representative and important is langchain @xcite , a python library for building llm applications supporting multiple paradigms including explicit chaining, agent-based modeling, and vector lookups. dust (dust) proposes a different toolkit approach using rust as a backend. programming languages and domain specific languages (dsls) to support programming with prompts. these include lmql @xcite a dsl for constraining model output, microsoft's semantic kernel @xcite , a heavy-weight toolkit supporting many different prompting paradigm across languages, and demonstrate-search-predict @xcite a dsl describing systems that integrate retrieval and llm decision making. collections of tools designed for llm usage. llama-index @xcite is a collection of data resources and software meant to help llms respond to targeted queries. other approaches focus on collecting additional models to consult, e.g. hug-ginggpt/jarvis @xcite , or open apis to utilize as in taskmatrix @xcite . llm toolkits designed to provide prompts for specific tasks. these libraries, such as promptify (pal, 2022), collect good versions of prompts that help solve specific zero-shot or few-shot tasks. these toolkits are less about the chaining component, but provide clear and usable prompt templates for the individual prompts. many of the other libraries also provide clear prompts as part of their system. ## programming with llms let us begin by considering a practical example of chaining language models. large language models have trouble with computing mathematical equations due to the limitations of fixed depth transformer models. for example, at the time of this writing asking google bard foot_0 to "sum the numbers 10 through 15" yields a confident assertion that the answer is 60. however, researchers have noted that they are extremely good at code generation, and can map natural language descriptions of math problems into usable code, e.g. @xcite . this code can then be run to produce an answer. this motivates the base case for chaining. given a word problem, we a) describe to the llm what we want to do and have it convert it to code, and then b) run the code in an interpreter to produce the result. we will think of both of these steps as "prompting", i.e. first prompt the llm and then prompt the interpreter. the minichain api has the user describe both of these steps using simple functions. first we describe how to ask to map the problem to code. @prompt(openai(), @xmath1.pmpt.tpl") def math_prompt(model, query): the function takes two arguments, a special model argument representing the llm and a user argument for the query to convert to code. the key additional component is the @prompt decorator which tells us which model to use (in this case the default openai model), as well as a template file with the details of the prompt to use. now let's look at the prompt itself stored as a jinja template 2 . the prompt is a few-shot description of the task itself. it contains several examples of questions and code outputs, as well as a template "hole" question to fill in with the user question before generating. (this format is inspired by the promptsource system ( bach et al., 2022 ## visualizing the llm interactions since the system uses lazily instantiated chain of prompts with explicit prompt templates, it has full transparency into each step of the prompting process. this design makes it easy to extract and control the intermediate states of the system before and during runtime. one particular benefit of this transparency is that it facilitates automatic interactive visualization and debugging. minichain includes a full visualization library built in based on the gradio visualization library foot_1 . the visualization does not require any additional code, beyond what was shown in the previous section, and can be launched with the following command. this command creates an automatic interactive visualization ui, which is shown in figure 1 . starting from a text input, it kicks off and runs each step of the lazy chain showing the intermediate steps and output from the system. the visualization shows each of the prompts, responses, model uses and the chain structure. expanding the '. . . ' will show additional low level information like the raw template, variables used, and the commands for calling the underlying llm model. the visualization mode of the library also supports additional modalities. minichain supports the ability to utilize models that are non-text based, such as images, video, and audio. in figure 2 , we have the model first write a story and then chain that with a stable diffusion model that draws the output into an image. in addition to being lazy, the graph used in minichain is by default streaming. this means that the visualization can display partial outputs from a call to an llm in real-time. for a slow model like gpt-4, showing intermediate results as they are generating presents a much improved user experience. practically this is configured through a customization of the prompt call. for example, in one of the examples we extract a table from a document as a csv file. this code will convert the output to a well-formatted html minichain is an opinionated library, and one of the goals is to not build additional features that are not related to chaining into the library. we argue for this minimality, by showing how popular prompt paradigms can be implemented without custom support. ## conclusion we describe minichain, a software toolkit for prompt chaining. the library focuses on creating an explicit chain of prompts each of which are simple python functions. each prompt separates out the descriptive language from the actual chain logic, and transformation logic can be specified with standard python code. the system is transparent, which allows automatic interactive visualization with different modalities, streaming, and detailed debugging. the core language is minimal, but powerful enough to implement core prompt paradigms, such as retrieval, chat-bots, and tooluse. we hope this work demonstrates some of the possibilities of prompt programming, and encourages others to think about the apis of these systems and how llms will integrate into software.