Repository: hpe-dev-incubator/hpe-dev-portal
Pull request #2964: Create Blog “using-structured-outputs-in-vllm”
Author: login: ismael-dm
Created: 2025-03-16T19:33:30Z | Merged: 2025-03-19T12:07:46Z

Automatically generated by Netlify CMS


Diff:
diff --git a/content/blog/using-structured-outputs-in-vllm.md b/content/blog/using-structured-outputs-in-vllm.md
new file mode 100644
index 0000000000..9120a9880d
--- /dev/null
+++ b/content/blog/using-structured-outputs-in-vllm.md
@@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even formal grammars. vLLM, an open source inference and serving engine for LLMs, has supported structured outputs for a while. However, there is little documentation on how to use it. This is why I decided to contribute and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine there is an external system which receives a JSON object with the all the details to trigger an alert, and you want your LLM-based system to be able to use it. Of course you could try to explain the LLM what should be the output format and that it must be a valid JSON object, but LLMs are not deterministic and thus you may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How do these tools work? The idea behind them is to filter a list of possible next tokens to force a valid token to be generated that produces the desired output format, for example, a valid JSON object.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Guided choice is the simplest form of structured output. It ensures the response is one from of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+A guided regex constrains the output to match a regex pattern, which is useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Guided JSON enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided grammar**
+
+Guided grammar uses an Extended Backus-Naur Form (EBNF) grammar syntax to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next steps**
+
+To start integrating structured outputs into your projects:
+
+1. **Explore the documentation:** Check out the [official documentation](https://docs.vllm.ai/en/latest/) for more examples and detailed explanations.
+2. **Install vLLM locally:** Set up the inference server on your local machine using the [vLLM GitHub repository](https://github.com/vllm-project/vllm).
+3. **Experiment with structured outputs:** Try out different formats (choice, regex, JSON, grammar) and observe how they can simplify your workflow.
+4. **Deploy in production:** Once comfortable, deploy vLLM to your production environment and integrate it with your applications.
+
+Structured outputs make LLMs not only powerful but also practical for real-world applications. Dive in and see what you can build!
\ No newline at end of file
diff --git a/static/img/structured_outputs_thumbnail.png b/static/img/structured_outputs_thumbnail.png
new file mode 100644
index 0000000000..9be34f96d8
Binary files /dev/null and b/static/img/structured_outputs_thumbnail.png differ


Reviews:
user: login: choukden
body: Hi Ismael,

Thank you very much for your first contribution.
I have added a few suggestions for changes for your consideration. 

I will reach out to Dale Rensing, our editorial consultant, to review your blog post from an editorial standpoint. Please expect additional suggestions for changes through GitHub.

1. _For formatting the bullet items as standard text, here is the tip:_

Insert the following Style statement (this is the recommended style statement) in your markdown at **the beginning of your post** (in the Body). The standard text size is 27px.

<style>
li {
   font-size: 27px;
   line-height: 33px;
   max-width: none;
}
</style>
  
<p></p>

**IMPORTANT NOTE:** 
_The fonts may appear bigger than the standard text in the right panel of the CMS editor. This is expected. However, the Netlify process will render the text correctly (as standard text size) when creating the “Preview” of the post and when publishing the post to the HPE Developer Community portal._

2. _Here is another tip for review of the suggestions for changes and how to marked them as resolved:_

To review the suggestions and acknowledge them in GitHub:

- Navigate to **File Changed** area to visualize the comments from start to the end, or navigate to **Conversation** area to visualize the comments per reviewer.

- You can mark the suggestion you have incorporated in your platform page entry in the CMS Workflow Editor by clicking on button **Resolve Conversation.**

**Note:** we recommend you continue doing edits of your platform entry in the CMS Workflow editor while the entry is in the "**In Review**" column. Moving it back to the "**Drafts**” column will close the Pull Request (PR) in GitHub and when moving the entry to the “**In Review**” column _**again**_ a new PR in GitHub is created, and we will lose the initial suggestions for edits.

Hope you will find the suggestions for changes and the tips above helpful and useful.

Best regards,

Denis
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T12:42:43Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:39:59Z
user: login: choukden
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T14:54:28Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:57:25Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T14:57:51Z
user: login: drensing
body: Hi there. Thank you very much for your submission. I hope you find my suggested edits helpful.

All the best,
Dale
state: CHANGES_REQUESTED
author_association: COLLABORATOR
submitted_at: 2025-03-17T19:23:31Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T19:30:00Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:12:03Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:14:43Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:16:14Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:17:30Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:18:45Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:19:59Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:20:46Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:21:13Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:21:53Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T20:21:59Z
user: login: drensing
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-18T20:28:43Z
user: login: drensing
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-18T20:29:03Z
user: login: choukden
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-19T11:50:46Z
user: login: ismael-dm
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T11:52:43Z

Review comments:
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Not al readers are familiar with vLLM.
Suggest to slightly change the 3rd sentence to spell out vLLM the first time you mention it in the post:

Virtual large language model (vLLM), a library of open source code, supports structured outputs since some time ago, but there were no documentation on how to use it,....

created_at: 2025-03-17T12:00:57Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: At the end of the introductory paragraph, I would recommend to add a short sentence to ensure you grab the reader’s attention.

Add a line space, and you can add something like that:

In this blog post, I'll explain....

Dale Rensing, our editorial consultant will probably help you with the sentence to grab readers attention. 
created_at: 2025-03-17T12:05:25Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: HPE Brand is requesting to use sentence case:

## Why structured outputs?
created_at: 2025-03-17T12:06:36Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: In the picture, you mention VLM model...shouldn't it be vLLM model?
created_at: 2025-03-17T12:09:21Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Continuous batching:**
created_at: 2025-03-17T12:11:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Advanced optimizations:**
created_at: 2025-03-17T12:11:46Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: ### **1. Guided choice**
created_at: 2025-03-17T12:14:15Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Example output:**
created_at: 2025-03-17T12:14:38Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Example output:**
created_at: 2025-03-17T12:14:58Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Example output:**
created_at: 2025-03-17T12:15:18Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: ### **4. Guided grammar**
created_at: 2025-03-17T12:15:50Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Suggest to spell out EBNF the first time you mention it in the post:

Used an extended Backus–Naur form (EBNF), a way to specify a formal language grammar, to define complex output structures, such as SQL queries.
created_at: 2025-03-17T12:18:14Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: **Example output:**
created_at: 2025-03-17T12:18:31Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next Steps**
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: ## **Next steps**
created_at: 2025-03-17T12:18:47Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next Steps**
+
+To start integrating structured outputs into your projects:
+
+1. **Explore the Documentation:** Check out the official documentation for more examples and detailed explanations.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Sentence case:
**Explore the documentation:**

Also, I suggest to add a hotlink to the "Explore the documentation:" Check out the [official documentation](HTTP URL to the doc) for more examples and detailed explanations.
created_at: 2025-03-17T12:28:06Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next Steps**
+
+To start integrating structured outputs into your projects:
+
+1. **Explore the Documentation:** Check out the official documentation for more examples and detailed explanations.
+2. **Install vLLM Locally:** Set up the inference server on your local machine using the vLLM GitHub repository.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Sentence case:
**Install vLLM locally:** 

And add hotlink to GitHub repo:
Set up the inference server on your local machine using the [vLLM GitHub repository](URL to GitHub repo].
created_at: 2025-03-17T12:29:13Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next Steps**
+
+To start integrating structured outputs into your projects:
+
+1. **Explore the Documentation:** Check out the official documentation for more examples and detailed explanations.
+2. **Install vLLM Locally:** Set up the inference server on your local machine using the vLLM GitHub repository.
+3. **Experiment with Structured Outputs:** Try out different formats (choice, regex, JSON, grammar) and observe how they can simplify your workflow.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Sentence case:
**Experiment with structured outputs:** 
created_at: 2025-03-17T12:29:41Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
+* **Advanced Optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided Choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided Grammar**
+
+Uses an EBNF grammar to define complex output structures, such as SQL queries.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a SQL query to find all users older than 30."}
+    ],
+    extra_body={
+        "guided_grammar": """
+        query ::= "SELECT" fields "FROM users WHERE" condition;
+        fields ::= "name, age" | "*";
+        condition ::= "age >" number;
+        number ::= [0-9]+;
+        """
+    },
+)
+print(completion.choices[0].message.content)
+```
+
+**Example Output:**
+
+```sql
+SELECT * FROM users WHERE age > 30;
+```
+
+## **Next Steps**
+
+To start integrating structured outputs into your projects:
+
+1. **Explore the Documentation:** Check out the official documentation for more examples and detailed explanations.
+2. **Install vLLM Locally:** Set up the inference server on your local machine using the vLLM GitHub repository.
+3. **Experiment with Structured Outputs:** Try out different formats (choice, regex, JSON, grammar) and observe how they can simplify your workflow.
+4. **Deploy in Production:** Once comfortable, deploy vLLM to your production environment and integrate it with your applications.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Sentence case:

**Deploy in production:**
created_at: 2025-03-17T12:30:26Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: vLLM is just the name of the tool, it doesn´t have a spelling. Maybe I can put the link to vLLM github and the explanation in the 3rd form as you are saying. For example: "vLLM, a library of open source code, supports structured outputs since some time ago, but there were no documentation on how to use it,...." sounds good? 
created_at: 2025-03-17T14:39:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: Looks good to me. Denis.
created_at: 2025-03-17T14:54:28Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: What happens here? I see you marked multiple of lines like this, but I don´t understand the issue
created_at: 2025-03-17T14:57:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,179 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+## Why Structured Outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous Batching:** Supports concurrent requests dynamically.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: oh I see, the sentence case? 
created_at: 2025-03-17T14:57:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Hi there. I'm here to assist by reviewing the blog post for Brand compliance, grammar, and readability.

In your second sentence, you mention "enforcing specific formats, such as JSON, regex patterns, or even grammars." I'm not sure what you mean here and I'm not familiar enough with LLMs to know if they refer to something as "grammars". Might you actually mean "such as a JSON schema, regex patterns, or grammar and syntax."?

For the next few sentences, I recommend rephrasing to:

vLLM, an open source inference and serving engine for LLMs, has supported structured outputs for a while. However, there is little documentation on how to use it. This is why I decided to contribute and write the Structured Outputs documentation page.
created_at: 2025-03-17T19:05:15Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Interesting... this line doesn't show up in the preview. It's a good sentence.
![image](https://github.com/user-attachments/assets/31dfd893-c518-4235-8667-3320fc2a894d)


created_at: 2025-03-17T19:06:05Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Modify to:
"Imagine there is an external system that receives a JSON (a JSON what? a JSON output?) with all the details required to trigger an alert, and you want your LLM-based system to be able to use it."

Here, you are speaking directly to the reader and it helps provide clarity around who does what. Then, you would modify the rest to:

"Of course, you could try to specify that the LLM output format must be a valid JSON (are we missing a word after JSON?), but LLMs are not deterministic and thus you may end up with an invalid JSON."

It's highly recommended, btw, to avoid using what's called "the Royal 'We'". It tends to set up blog posts in a way that gets confusing the reader. It's better to think of yourself, the writer, as "I" and the reader as "you" and to stick with that so as to not confuse things.
created_at: 2025-03-17T19:12:40Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: add in the word "do"... How do these tools work?"

Consider rephrasing the next sentence to:

"The idea behind them is to filter a list of possible next tokens to force a valid token to be generated that produces the desired output format." or something similar. As it is currently written, it is confusing.
created_at: 2025-03-17T19:14:32Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Would it make sense to place this section previous to the section on Structured outputs?
created_at: 2025-03-17T19:15:40Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: So, if you started off with What is vLLM, then went to Structured outputs, this next section would naturally follow.
created_at: 2025-03-17T19:17:14Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Are you sure "grammars" is the right term? Should it possibly be "grammar"? In general, "grammar" is both singular and plural.
created_at: 2025-03-17T19:17:49Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Make this introductory sentence a complete sentence, such as:
**Guided choice is the** simplest form of structured output. It ensures the response is one **from** of a set of predefined options.
created_at: 2025-03-17T19:19:39Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Again, the first sentence ought to be a complete sentence. Try this:

A guided regex constrains the output to match a regex pattern, which is useful for formats like email addresses.
created_at: 2025-03-17T19:20:32Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Similarly, make this a complete sentence:
Guided JSON enforces a valid JSON format based on a schema, simplifying integration with other systems.
created_at: 2025-03-17T19:21:12Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided grammar**
+
+Uses an Extended Backus–Naur Form (EBNF) grammar to define complex output structures, such as SQL queries.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Guided grammar uses an Extended Backus-Naur Form (EBNF) grammar **syntax** to define complex output structures, such as SQL queries.

(though I'm not positive, I think you need a word after grammar here and syntax looks like it might possibly be the right word. )
created_at: 2025-03-17T19:22:39Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: I think I just added it before you open the page, so it was taking some time to render. Now I can see it from the preview
created_at: 2025-03-17T19:30:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: changing grammars to formal grammars
rephrased as suggested
created_at: 2025-03-18T20:12:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: great, changes applied
created_at: 2025-03-18T20:14:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: applied
created_at: 2025-03-18T20:16:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: maybe putting it before is too much information without introduction, I dont see it clear
created_at: 2025-03-18T20:17:30Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: so you mean this order? 
1. What is vllm? 
2. why structured outputs
3. structured outputs on vllm
is this correct? 
created_at: 2025-03-18T20:18:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: this is normally referred as grammars also in other tools like outlines (a tool dedicated to define grammars for llms: https://github.com/dottxt-ai/outlines)
created_at: 2025-03-18T20:19:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: applied
created_at: 2025-03-18T20:20:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: applied
created_at: 2025-03-18T20:21:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
+
+```python
+from pydantic import BaseModel
+from enum import Enum
+
+class CarType(str, Enum):
+    sedan = "sedan"
+    suv = "SUV"
+    truck = "Truck"
+    coupe = "Coupe"
+
+class CarDescription(BaseModel):
+    brand: str
+    model: str
+    car_type: CarType
+
+json_schema = CarDescription.model_json_schema()
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Generate a JSON for the most iconic car from the 90s."}
+    ],
+    extra_body={"guided_json": json_schema},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```json
+{
+  "brand": "Toyota",
+  "model": "Supra",
+  "car_type": "coupe"
+}
+```
+
+### **4. Guided grammar**
+
+Uses an Extended Backus–Naur Form (EBNF) grammar to define complex output structures, such as SQL queries.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: applied
created_at: 2025-03-18T20:21:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.
+
+Here’s how each works, along with example outputs:
+
+### **1. Guided choice**
+
+Simplest form of structured output, ensuring the response is one of a set of predefined options.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
+
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {"role": "user", "content": "Classify this sentiment: vLLM is wonderful!"}
+    ],
+    extra_body={"guided_choice": ["positive", "negative"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+positive
+```
+
+### **2. Guided Regex**
+
+Constrains output to match a regex pattern, useful for formats like email addresses.
+
+```python
+completion = client.chat.completions.create(
+    model="Qwen/Qwen2.5-3B-Instruct",
+    messages=[
+        {
+            "role": "user",
+            "content": "Generate an example email address for Alan Turing at Enigma. End in .com.",
+        }
+    ],
+    extra_body={"guided_regex": r"\w+@\w+\.com\n", "stop": ["\n"]},
+)
+print(completion.choices[0].message.content)
+```
+
+**Example output:**
+
+```
+alan.turing@enigma.com
+```
+
+### **3. Guided JSON**
+
+Enforces a valid JSON format based on a schema, simplifying integration with other systems.
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: applied
created_at: 2025-03-18T20:21:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: Yes. That flow feels like it works better for me. But it's up to you. As the author, you know better what you're trying to get across. We could also check with Denis to get his opinion. 
created_at: 2025-03-18T20:28:43Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: drensing
body: @choukden please see above.
created_at: 2025-03-18T20:29:03Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: choukden
body: @ismael-dm ; @drensing 

I would let Ismael decide on the appropriate structure. Both content structures look good to me. 
I can merge the PR and publish the Post.

Thanks

Denis  
created_at: 2025-03-19T11:50:46Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,181 @@
+---
+title: Using structured outputs in vLLM
+date: 2025-03-16T19:28:00.657Z
+author: Ismael Delgado Muñoz
+authorimage: /img/Avatar6.svg
+thumbnailimage: ""
+disable: false
+tags:
+  - AI
+  - GenAI
+  - opensource
+  - LLM
+---
+<style> li { font-size: 27px; line-height: 33px; max-width: none; } </style>
+Generating predictable and reliable outputs from large language models (LLMs) can be challenging, especially when those outputs need to integrate seamlessly with downstream systems. Structured outputs solve this problem by enforcing specific formats, such as JSON, regex patterns, or even grammars. vLLM, an open source inference and serving engine for LLMs, supports structured outputs since some time ago, but there were no documentation on how to use it supported this since some time ago, but there were no documentation on how to use it, and that´s why I decided to do a contribution and write the [Structured Outputs documentation page](https://docs.vllm.ai/en/latest/usage/structured_outputs.html).
+
+In this blog post, I'll explain how structured outputs work in vLLM and walk you through how to use them effectively.
+
+## Why structured outputs?
+
+LLMs are incredibly powerful, but their outputs can be inconsistent when a specific format is required. Structured outputs address this issue by restricting the model’s generated text to adhere to predefined rules or formats, ensuring:
+
+1. **Reliability:** Outputs are predictable and machine-readable.
+2. **Compatibility:** Seamless integration with APIs, databases, or other systems.
+3. **Efficiency:** No need for extensive post-processing to validate or fix outputs.
+
+Imagine we have an external system which receives a JSON with the all the details to trigger an alert, and we want our LLM-based system to be able to use it. Of course we can try to explain the LLM what should be the output format and that it must be a valid JSON, but LLMs are not deterministic and thus we may end up with an invalid JSON. Probably, if you have tried to do something like this before, you would have found yourself in this situation.
+
+How these tools work? The idea is that we´ll be able to filter the list of possible next tokens to force that we are always generating a token that is valid for the desired output format.
+
+![Structured outputs in vLLM](/img/structured_outputs_thumbnail.png "Structured outputs in vLLM")
+
+## What is vLLM?
+
+vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
path: content/blog/using-structured-outputs-in-vllm.md
user: login: ismael-dm
body: so current status looks good to me, if you agree we can close and publish
created_at: 2025-03-19T11:52:43Z
author_association: CONTRIBUTOR

PR comments:
user: login: netlify[bot]
created_at: 2025-03-16T19:33:48Z
body: ### <span aria-hidden="true">✅</span> Deploy Preview for *hpe-dev-portal* ready!
Built [without sensitive environment variables](https://docs.netlify.com/configure-builds/environment-variables/#sensitive-variable-policy)

|  Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | b091df47050d9d31be23e14110582313625b196d |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/hpe-dev-portal/deploys/67d9e7db008613000855fc8a |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-2964--hpe-dev-portal.netlify.app](https://deploy-preview-2964--hpe-dev-portal.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br />![QR Code](https://app.netlify.com/qr-code/eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cmwiOiJodHRwczovL2RlcGxveS1wcmV2aWV3LTI5NjQtLWhwZS1kZXYtcG9ydGFsLm5ldGxpZnkuYXBwIn0.uPJgN7cyiFKAKEBQNKkIH5q1MG-ZJAj9NhQuSFJtChc)<br /><br />_Use your smartphone camera to open QR code link._</details> |
---
<!-- [hpe-dev-portal Preview](https://deploy-preview-2964--hpe-dev-portal.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify site configuration](https://app.netlify.com/sites/hpe-dev-portal/configuration/notifications#deploy-webhooks)._
author_association: NONE
user: login: aws-amplify-us-east-1[bot]
created_at: 2025-03-16T19:52:03Z
body: This pull request is automatically being deployed by Amplify Hosting ([learn more](https://aws.amazon.com/amplify/hosting/)).<br/><br/>Access this pull request here: https://pr-2964.d2wz01xj8bm9sl.amplifyapp.com
author_association: NONE
user: login: drensing
created_at: 2025-03-18T20:27:15Z
body: [like]  Rensing, Dale reacted to your message:
________________________________
From: Ismael Delgado ***@***.***>
Sent: Tuesday, March 18, 2025 8:20:21 PM
To: hpe-dev-incubator/hpe-dev-portal ***@***.***>
Cc: Rensing, Dale ***@***.***>; Review requested ***@***.***>
Subject: Re: [hpe-dev-incubator/hpe-dev-portal] Create Blog “using-structured-outputs-in-vllm” (PR #2964)


@ismael-dm commented on this pull request.

________________________________

In content/blog/using-structured-outputs-in-vllm.md<https://github.com/hpe-dev-incubator/hpe-dev-portal/pull/2964#discussion_r2001923388>:

> +vLLM is a state-of-the-art, open-source inference and serving engine for LLMs. It’s built for performance and simplicity, offering:
+
+* **PagedAttention:** An innovative memory management mechanism for efficient attention key-value handling.
+* **Continuous batching:** Supports concurrent requests dynamically.
+* **Advanced optimizations:** Includes features like quantization, speculative decoding, and CUDA graphs.
+
+These optimizations make vLLM one of the fastest and most versatile engines for production environments.
+
+## Structured outputs on vLLM
+
+vLLM extends the OpenAI API with additional parameters to enable structured outputs. These include:
+
+* **guided_choice:** Restricts output to a set of predefined choices.
+* **guided_regex:** Ensures outputs match a given regex pattern.
+* **guided_json:** Validates outputs against a JSON schema.
+* **guided_grammar:** Enforces structure using context-free grammars.


this is normally referred as grammars also in other tools like outlines (a tool dedicated to define grammars for llms: https://github.com/dottxt-ai/outlines)

—
Reply to this email directly, view it on GitHub<https://github.com/hpe-dev-incubator/hpe-dev-portal/pull/2964#discussion_r2001923388>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ASPH7RMSKUHFWFUY7RQK5PT2VB5YLAVCNFSM6AAAAABZD7C7YCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDMOJWGEZTSMBWGA>.
You are receiving this because your review was requested.Message ID: ***@***.***>

author_association: COLLABORATOR