Package: outlines
Outlines 〰️Robust (structured) text generation.🙏 Help us by answering thedeveloper survey🙏Made with ❤👷️ by the team at.txt.Looking for an API that returns valid JSON?Give .json a try✨pipinstalloutlinesFirst time here? Go to oursetup guideFeatures🤖Multiple model integrations: OpenAI, transformers, llama.cpp, exllama2, mamba🖍️ Simple and powerful prompting primitives based on theJinja templating engine🚄Multiple choices,type constraintsand dynamic stopping⚡ Fastregex-structured generation🔥 FastJSON generationfollowing a JSON schema or a Pydantic model📝Grammar-structured generation🐍 Interleave completions with loops, conditionals, and custom Python functions💾 Caching of generations🗂️ Batch inference🎲 Sample with the greedy, multinomial and beam search algorithms (and more to come!)🚀Serve with vLLM, with official Docker image,outlinesdev/outlines!Outlines 〰 has new releases and features coming every week. Make sure to ⭐ star and 👀 watch this repository, follow@dottxtaito stay up to date!Structured generationThe first step towards reliability of systems that include large language models
is to ensure that there is a well-defined interface between their output and
user-defined code.Outlinesprovides ways to control the generation of
language models to make their output more predictable.Multiple choicesYou can reduce the completion to a choice between multiple possibilities:importoutlinesmodel=outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")prompt="""You are a sentiment-labelling assistant.Is the following review positive or negative?Review: This restaurant is just awesome!"""generator=outlines.generate.choice(model,["Positive","Negative"])answer=generator(prompt)Type constraintYou can instruct the model to only return integers or floats:importoutlinesmodel=outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")prompt="
result of 9 + 9 = 18

result of 1 + 2 = "answer=outlines.generate.format(model,int)(prompt)print(answer)# 3prompt="sqrt(2)="generator=outlines.generate.format(model,float)answer=generator(prompt,max_tokens=10)print(answer)# 1.41421356Efficient regex-structured generationOutlines also comes with fast regex-structured generation. In fact, thechoiceandformatfunctions above all use regex-structured generation under the
hood:importoutlinesmodel=outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")prompt="What is the IP address of the Google DNS servers? "generator=outlines.generate.text(model)unstructured=generator(prompt,max_tokens=30)generator=outlines.generate.regex(model,r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)",)structured=generator(prompt,max_tokens=30)print(unstructured)# What is the IP address of the Google DNS servers?## Passive DNS servers are at DNS servers that are private.# In other words, both IP servers are private. The database# does not contain Chelsea Manningprint(structured)# What is the IP address of the Google DNS servers?# 2.2.6.1Unlike other libraries, regex-structured generation in Outlines is almost as fast
as non-structured generation.Efficient JSON generation following a Pydantic modelOutlines 〰 allows to guide the generation process so the output isguaranteedto follow aJSON schemaorPydantic model:fromenumimportEnumfrompydanticimportBaseModel,constrimportoutlinesimporttorchclassWeapon(str,Enum):sword="sword"axe="axe"mace="mace"spear="spear"bow="bow"crossbow="crossbow"classArmor(str,Enum):leather="leather"chainmail="chainmail"plate="plate"classCharacter(BaseModel):name:constr(max_length=10)age:intarmor:Armorweapon:Weaponstrength:intmodel=outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")# Construct structured sequence generatorgenerator=outlines.generate.json(model,Character)# Draw a samplerng=torch.Generator(device="cuda")rng.manual_seed(789001)character=generator("Give me a character description",rng=rng)print(repr(character))# Character(name='Anderson', age=28, armor=<Armor.chainmail: 'chainmail'>, weapon=<Weapon.sword: 'sword'>, strength=8)character=generator("Give me an interesting character description",rng=rng)print(repr(character))# Character(name='Vivian Thr', age=44, armor=<Armor.plate: 'plate'>, weapon=<Weapon.crossbow: 'crossbow'>, strength=125)The method works with union types, optional types, arrays, nested schemas, etc. Some field constraints arenot supported yet, but everything else should work.Efficient JSON generation following a JSON SchemaSometimes you just want to be able to pass a JSON Schema instead of a Pydantic model. We've got you covered:importoutlinesschema='''{"title": "Character","type": "object","properties": {"name": {"title": "Name","maxLength": 10,"type": "string"},"age": {"title": "Age","type": "integer"},"armor": {"$ref": "#/definitions/Armor"},"weapon": {"$ref": "#/definitions/Weapon"},"strength": {"title": "Strength","type": "integer"}},"required": ["name", "age", "armor", "weapon", "strength"],"definitions": {"Armor": {"title": "Armor","description": "An enumeration.","enum": ["leather", "chainmail", "plate"],"type": "string"},"Weapon": {"title": "Weapon","description": "An enumeration.","enum": ["sword", "axe", "mace", "spear", "bow", "crossbow"],"type": "string"}}}'''model=outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")generator=outlines.generate.json(model,schema)character=generator("Give me a character description")Using context-free grammars to guide generationFormal grammars rule the world, and Outlines makes them rule LLMs too. You can pass any context-free grammar in the EBNF format and Outlines will generate an output that is valid to this grammar:importoutlinesarithmetic_grammar="""?start: expression?expression: term (("+" | "-") term)*?term: factor (("*" | "/") factor)*?factor: NUMBER| "-" factor| "(" expression ")"%import common.NUMBER"""model=outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")generator=outlines.generate.cfg(model,arithmetic_grammar)sequence=generator("Alice had 4 apples and Bob ate 2. Write an expression for Alice's apples:")print(sequence)# (8-2)This was a very simple grammar, and you can useoutlines.generate.cfgto generate syntactically valid Python, SQL, and much more than this. Any kind of structured text, really. All you have to do is search for "X EBNF grammar" on the web, and take a look at theOutlinesgrammarsmodule.Open functionsOutlines can infer the structure of the output from the signature of a function. The result is a dictionary, and can be passed directly to the function using the usual dictionary expansion syntax**:importoutlinesdefadd(a:int,b:int):returna+bmodel=outlines.models.transformers("WizardLM/WizardMath-7B-V1.1")generator=outlines.generate.json(model,add)result=generator("Return json with two integers named a and b respectively. a is odd and b even.")print(add(**result))# 3A great advantage of passing functions directly to specify the structure is that the structure of the LLM will change with the function's definition. No need to change the code at several places!PromptingBuilding prompts can get messy.Outlinesmakes it easier to write and manage
prompts by encapsulating templates inside "template functions".These functions make it possible to neatly separate the prompt logic from the
general program logic; they can be imported from other modules and libraries.Template functions require no superfluous abstraction, they use the Jinja2
templating engine to help build complex prompts in a concise manner:importoutlinesexamples=[("The food was disgusting","Negative"),("We had a fantastic night","Positive"),("Recommended","Positive"),("The waiter was rude","Negative")]@outlines.promptdeflabelling(to_label,examples):"""You are a sentiment-labelling assistant.{% for example in examples %}{{ example[0] }} // {{ example[1] }}{% endfor %}{{ to_label }} //"""model=outlines.models.transformers("mistralai/Mistral-7B-Instruct-v0.2")prompt=labelling("Just awesome",examples)answer=outlines.generate.text(model)(prompt,max_tokens=100)Join us💡Have an idea?Come chat with us onDiscord🔨Want to contribute?Consult ourcontribution guide.🐞Found a bug?Open anissue