Package: prompt-engine-py
Prompt EngineThis repo contains a Python utility library for creating and maintaining prompts for Large Language Models (LLMs).BackgroundLLMs like GPT-3 and Codex have continued to push the bounds of what AI is capable of - they can capably generate language and code, but are also capable of emergent behavior like question answering, summarization, classification and dialog. One of the best techniques for enabling specific behavior out of LLMs is called prompt engineering - crafting inputs that coax the model to produce certain kinds of outputs. Few-shot prompting is the discipline of giving examples of inputs and outputs, such that the model has a reference for the type of output you're looking for.Prompt engineering can be as simple as formatting a question and passing it to the model, but it can also get quite complex - requiring substantial code to manipulate and update strings. This library aims to make that easier. It also aims to codify patterns and practices around prompt engineering.SeeHow to get Codex to produce the code you wantarticle for an example of the prompt engineering patterns this library codifies.Installationpip install prompt-engine-pyUsageThe library currently supports a genericPromptEngine, aCodeEngineand aChatEngine. All three facilitate a pattern of prompt engineering where the prompt is composed of a description, examples of inputs and outputs and an ongoing "dialog" representing the ongoing input/output pairs as the user and model communicate. The dialog ensures that the model (which is stateless) has the context about what's happened in the conversation so far.See architecture diagram representation:Code EngineCode Engine creates prompts for Natural Language to Code scenarios. See Python Syntax for importingCodeEngineandPythonCodeEngineConfig:fromprompt_engine.code_engineimportCodeEngine,PythonCodeEngineConfigNL->Code prompts should generally have a description, which should give context about the programming language the model should generate and libraries it should be using. The description should also give information about the task at hand:description="Natural Language Commands to JavaScript Math Code. The code should log the result of the command to the console."NL->Code prompts should also have examples of NL->Code interactions, exemplifying the kind of code you expect the model to produce. In this case, the inputs are math queries (e.g. "what is 2 + 2?") and code that console logs the result of the query.fromprompt_engine.interactionimportInteractionexamples=[Interaction("what's 10 plus 18","console.log(10 + 18)"),Interaction("what's 10 times 18","console.log(10 * 18)")]By default,CodeEngineuses Python as the programming language, but you can create prompts for different languages by passing a differentCodeEngineConfiginto the constructor. If, for example, we wanted to produce JavaScript prompts, we could have passedCodeEngineajavascript_configspecifying the comment operator it should be using:javascript_config=CodeEngineConfig(description_comment_operator="/*/",description_comment_close_operator="/*/",comment_operator="/*",comment_close_operator="*/")code_engine=CodeEngine(config=javascript_config,description=description,examples=examples)With our description and our examples, we can use it to create prompts:query="What's 1018 times the ninth power of four?"prompt=code_engine.build_prompt(query)The resulting prompt will be a string with the description, examples and the latest query formatted with comment operators and line breaks:/*/ Natural Language Commands to JavaScript Math Code. The code should log the result of the command to the console. /*//* what's 10 plus 18 */console.log(10+18)/* what's 10 times 18 */console.log(10*18)/* What's 1018 times the ninth power of four? */Given the context, a capable code generation model can take the above prompt and guess the next line:print(1018 * (4 ** 9)).For multi-turn scenarios, where past conversations influences the next turn, Code Engine enables us to persist interactions in a prompt:...# Assumes existence of code generation modelcode=model.generate_code(prompt)# Adds interactioncode_engine.add_interaction(query,code)Now new prompts will include the latest NL->Code interaction:code_engine.build_prompt("How about the 8th power?")Produces a prompt identical to the one above, but with the NL->Code dialog history:.../* What's 1018 times the ninth power of four? */console.log(1018*(4**9))/* How about the 8th power? */With this context, the code generation model has the dialog context needed to understand what we mean by the query. In this case, the model would correctly generateprint(1018 * (4 ** 8)).Chat EngineJust like Code Engine, Chat Engine creates prompts with descriptions and examples. See Python Syntax for importingCodeEngineandPythonCodeEngineConfig:fromprompt_engine.chat_engineimportChatEngine,ChatEngineConfigThe difference is that Chat Engine creates prompts for dialog scenarios, where both the user and the model use natural language. TheChatEngineconstructor takes an optionalconfigargument, which allows you to define the name of a user and chatbot in a multi-turn dialog:config=ChatEngineConfig(user_name="Abhishek",bot_name="Gordon")Chat prompts also benefit from a description that gives context. This description helps the model determine how the bot should respond.description="A conversation with Gordon the Anxious Robot. Gordon tends to reply nervously and asks a lot of follow-up questions."Similarly, Chat Engine prompts can have examples interactions:fromprompt_engine.interactionimportInteractionexamples=[Interaction("Who made you?","I don't know man! That's an awfully existential question. How would you answer it?"),Interaction("Good point - do you at least know what you were made for?","I'm OK at riveting, but that's not how I should answer a meaning of life question is it?")]These examples help set the tone of the bot, in this case Gordon the Anxious Robot. Now we can create ourChatEngineand use it to create prompts:chat_engine=ChatEngine(chatEngineConfig,description,examples)user_query="What are you made of?"prompt=chat_engine.build_prompt(user_query)When passed to a large language model (e.g. GPT-3), the context of the above prompt will help coax a good Marvin-like answer from the model, like "Subatomic particles at some level, but somehow I don't think that's what you were asking.". As with Code Engine, we can persist this answer and continue the dialog such that the model is aware of the conversation context:chatEngine.add_interaction(user_query,"Subatomic particles at some level, but somehow I don't think that's what you were asking.")Dynamic Prompt EngineDynamic Prompt Engine is another behaviour constructed on top of prompt engine that enables dynamic retrieval of the relevant examples in order to generate a prompt.It is developed with the belief that giving more relevant examples to the Large Language Model will enable it to generate a better output and much closer to our examples, rather than leaving it to guess for itself. This also allows the ability to coax multiple behaviours out of the Large Language Model instead of having to maintain mutliple different prompts.The dynamic prompt engine maintains a prompt bank, which is a collection of embeddings of all the examples and interactions that have been provided to it. When given a new unseen prompt, it queries the prompt bank based on the embeddings to retrieve the Top-k relevant examples and adds them to the examples section of the prompt engine output.Managing Prompt OverflowPrompts for Large Language Models generally have limited size, depending on the language model being used. Given that prompt-engine can persist dialog history, it is possible for dialogs to get so long that the prompt overflows. The Prompt Engine pattern handles this situation by removing the oldest dialog interaction from the prompt, effectively only remembering the most recent interactions.You can specify the maximum tokens allowed in your prompt by passing amax_tokensparameter when constructing the config for any prompt engine:fromprompt_engine.model_configimportModelConfigconfig=PromptEngineConfig(ModelConfig(max_tokens=1024))Available FunctionsThe following are the functions available on thePromptEngineclass and those that inherit from it:CommandParametersDescriptionReturnsbuild_contextNoneConstructs and return the context with parameters provided to the Prompt EngineContext: stringbuild_promptPrompt: stringCombines the context frombuild_contextwith a query to create a promptPrompt: stringbuild_dialogNoneBuilds a dialog based on all the past interactions added to the Prompt EngineDialog: stringadd_exampleinteraction: Interaction(input: string, response: string)Adds the given example to the examplesNoneadd_interactioninteraction: Interaction(input: string, response: string)Adds the given interaction to the dialogNoneremove_first_interactionNoneRemoves and returns the first interaction in the dialogInteraction: Interactionremove_last_interactionNoneRemoves and returns the last interaction added to the dialogInteraction: Interactionreset_contextNoneRemoves all interactions from the dialog, effectively resetting the context to just description and examplesContext: stringFor more examples and insights into using the prompt-engine library, have a look at theexamplesfolderYAML RepresentationIt can be useful to represent prompts as standalone files, versus code. This can allow easy swapping between different prompts, prompt versioning, and other advanced capabiliites. With this in mind, prompt-engine offers a way to represent prompts as YAML and to load that YAML into a prompt-engine class. Seeexamples/yaml-examplesfor examples of YAML prompts and how they're loaded into prompt-engine.ContributingThis project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visithttps://cla.opensource.microsoft.com.When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.Statement of PurposeThis library aims to simplify use of Large Language Models, and to make it easy for developers to take advantage of existing patterns. The package is released in conjunction with theBuild 2022 AI examples, as the first three use a multi-turn LLM pattern that this library simplifies. This package works independently of any specific LLM - prompt generated by the package should be useable with various language and code generating models.TrademarksThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.