Repository: run-llama/llama_index
Pull request #17913: Contextual Generate model
Author: login: sean-smith
Labels: name: size:L
description: This PR changes 100-499 lines, ignoring generated files.
Created: 2025-02-25T09:39:07Z | Merged: 2025-03-12T11:11:14Z

# Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

## New Package?

Did I fill in the `tool.llamahub` section in the `pyproject.toml` and provide a detailed README.md for my new integration or package?

- [X] Yes
- [ ] No

## Version Bump?

Did I bump the version in the `pyproject.toml` file of the package I am updating? (Except for the `llama-index-core` package)

- [ ] Yes
- [X] No

## Type of Change

Please delete options that are not relevant.

- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [x] This change requires a documentation update

## How Has This Been Tested?

Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.

- [x] I added new unit tests to cover this change
- [ ] I believe this change is already covered by existing unit tests

## Suggested Checklist:

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] I have added Google Colab support for the newly added notebooks.
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I ran `make format; make lint` to appease the lint gods



Diff:
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/BUILD b/llama-index-integrations/llms/llama-index-llms-contextual/BUILD
new file mode 100644
index 00000000000..0896ca890d8
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/BUILD
@@ -0,0 +1,3 @@
+poetry_requirements(
+    name="poetry",
+)
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/README.md b/llama-index-integrations/llms/llama-index-llms-contextual/README.md
new file mode 100644
index 00000000000..2957768c86d
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/README.md
@@ -0,0 +1,19 @@
+# Contextual LLM Integration for LlamaIndex
+
+This package provides a Contextual LLM integration for LlamaIndex.
+
+## Installation
+
+```bash
+pip install llama-index-llms-contextual
+```
+
+## Usage
+
+```python
+from llama_index.llms.contextual import Contextual
+
+llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+response = llm.complete("Explain the importance of Grounded Language Models.")
+```
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/BUILD b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/BUILD
new file mode 100644
index 00000000000..db46e8d6c97
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/BUILD
@@ -0,0 +1 @@
+python_sources()
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/__init__.py b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/__init__.py
new file mode 100644
index 00000000000..42b34670f9e
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/__init__.py
@@ -0,0 +1,3 @@
+from llama_index.llms.contextual.base import Contextual
+
+__all__ = ["Contextual"]
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
new file mode 100644
index 00000000000..fe3ec581960
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
@@ -0,0 +1,187 @@
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from typing import List
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    ChatResponse,
+    ChatResponseGen,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(
+        description="The model to use. Currently only supports `v1`.", default="v1"
+    )
+    api_key: str = Field(description="The API key to use.", default=None)
+    base_url: str = Field(
+        description="The base URL to use.",
+        default="https://api.contextual.ai/v1/generate",
+    )
+    avoid_commentary: bool = Field(
+        description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.",
+        default=False,
+    )
+    client: Any = Field(default=None, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str,
+        base_url: str = None,
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Error initializing ContextualAI client: {e}")
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(
+        self, prompt: str, knowledge: Optional[List[str]] = None, **kwargs
+    ) -> CompletionResponse:
+        """
+        Generate completion for the given prompt.
+
+        Args:
+            prompt (str): The input prompt to generate completion for.
+            **kwargs: Additional keyword arguments for the API request.
+
+        Returns:
+            str: The generated text completion.
+        """
+        messages_list = [{"role": MessageRole.USER, "content": prompt}]
+        response = self._generate(
+            knowledge=knowledge,
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return CompletionResponse(text=response)
+
+    @llm_chat_callback()
+    def chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponse:
+        """
+        Generate a chat response for the given messages.
+        """
+        messages_list = [
+            {"role": msg.role, "content": msg.blocks[0].text} for msg in messages
+        ]
+        response = self._generate(
+            knowledge=kwargs.get("knowledge_base", None),
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return ChatResponse(
+            message=ChatMessage(role=MessageRole.ASSISTANT, content=response)
+        )
+
+    @llm_chat_callback()
+    def stream_chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    def stream_complete(self, prompt: str, **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    # ===== Async Endpoints =====
+    @llm_chat_callback()
+    async def achat(
+        self,
+        messages: Sequence[ChatMessage],
+        **kwargs: Any,
+    ) -> ChatResponse:
+        raise NotImplementedError("async methods not implemented in Contextual")
+
+    @llm_chat_callback()
+    async def astream_chat(
+        self,
+        messages: Sequence[ChatMessage],
+        **kwargs: Any,
+    ) -> ChatResponseAsyncGen:
+        raise NotImplementedError("async methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    async def acomplete(
+        self, prompt: str, formatted: bool = False, **kwargs: Any
+    ) -> CompletionResponse:
+        raise NotImplementedError("async methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    async def astream_complete(
+        self, prompt: str, formatted: bool = False, **kwargs: Any
+    ) -> CompletionResponseAsyncGen:
+        raise NotImplementedError("async methods not implemented in Contextual")
+
+    def _generate(
+        self, knowledge, messages, system_prompt, **kwargs
+    ) -> CompletionResponse:
+        """
+        Generate completion for the given prompt.
+        """
+        raw_message = self.client.generate.create(
+            messages=messages,
+            knowledge=knowledge or [],
+            model=self.model,
+            system_prompt=system_prompt,
+            avoid_commentary=self.avoid_commentary,
+            temperature=kwargs.get("temperature", 0.0),
+            max_new_tokens=kwargs.get("max_tokens", 1024),
+            top_p=kwargs.get("top_p", 1),
+        )
+        return raw_message.response
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/test-contextual.ipynb b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/test-contextual.ipynb
new file mode 100644
index 00000000000..a796fd1ebf5
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/test-contextual.ipynb
@@ -0,0 +1,146 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Contextual GLM"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install llama-index-llms-contextual"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CompletionResponse(text=\"I apologize, but I am unable to provide information about Grounded Language Models. I am an AI assistant created by Contextual AI. I don't have relevant documentation about that topic, but feel free to ask me something else!\", additional_kwargs={}, raw=None, logprobs=None, delta=None)"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from llama_index.llms.contextual import Contextual\n",
+    "from dotenv import load_dotenv\n",
+    "import os\n",
+    "\n",
+    "# Set up the Contextual class with the required model and API key\n",
+    "# Store the API key in a .env file as CONTEXTUAL_API_KEY\n",
+    "load_dotenv()\n",
+    "llm = Contextual(model=\"v1\", api_key=os.getenv(\"CONTEXTUAL_API_KEY\"))\n",
+    "\n",
+    "# Call the complete method with a query\n",
+    "llm.complete(\n",
+    "    \"Explain the importance of Grounded Language Models.\",\n",
+    "    temperature=0.5,\n",
+    "    max_tokens=1024,\n",
+    "    top_p=0.9,\n",
+    "    avoid_commentary=False,\n",
+    "    system_prompt=\"You are a helpful assistant that can answer questions and help with tasks.\",\n",
+    "    knowledge=[\"The sky is blue\"],\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CompletionResponse(text='The sky is blue.', additional_kwargs={}, raw=None, logprobs=None, delta=None)"
+      ]
+     },
+     "execution_count": null,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm.complete(\n",
+    "    \"what color is the sky?\",\n",
+    "    knowledge=[\"The sky is blue\"],\n",
+    "    avoid_commentary=False,\n",
+    "    temperature=0.9,\n",
+    "    max_tokens=1,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "AuthenticationError",
+     "evalue": "Error code: 401 - {'error': {'message': 'Incorrect API key provided: key-KCj3*****************************************izwo. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mAuthenticationError\u001b[0m                       Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[39], line 3\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mllama_index\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mcore\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mchat_engine\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mtypes\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m ChatMessage\n\u001b[0;32m----> 3\u001b[0m \u001b[43mllm\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[43m(\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatMessage\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrole\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcontent\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mwhat color is the sky?\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/projects/llama_index/llama-index-core/llama_index/core/instrumentation/dispatcher.py:322\u001b[0m, in \u001b[0;36mDispatcher.span.<locals>.wrapper\u001b[0;34m(func, instance, args, kwargs)\u001b[0m\n\u001b[1;32m    319\u001b[0m             _logger\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mFailed to reset active_span_id: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m    321\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 322\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    323\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(result, asyncio\u001b[38;5;241m.\u001b[39mFuture):\n\u001b[1;32m    324\u001b[0m         \u001b[38;5;66;03m# If the result is a Future, wrap it\u001b[39;00m\n\u001b[1;32m    325\u001b[0m         new_future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mensure_future(result)\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/llama_index/llms/openai_like/base.py:117\u001b[0m, in \u001b[0;36mOpenAILike.chat\u001b[0;34m(self, messages, **kwargs)\u001b[0m\n\u001b[1;32m    114\u001b[0m     completion_response \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcomplete(prompt, formatted\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m    115\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m completion_response_to_chat_response(completion_response)\n\u001b[0;32m--> 117\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/projects/llama_index/llama-index-core/llama_index/core/instrumentation/dispatcher.py:322\u001b[0m, in \u001b[0;36mDispatcher.span.<locals>.wrapper\u001b[0;34m(func, instance, args, kwargs)\u001b[0m\n\u001b[1;32m    319\u001b[0m             _logger\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mFailed to reset active_span_id: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m    321\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 322\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    323\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(result, asyncio\u001b[38;5;241m.\u001b[39mFuture):\n\u001b[1;32m    324\u001b[0m         \u001b[38;5;66;03m# If the result is a Future, wrap it\u001b[39;00m\n\u001b[1;32m    325\u001b[0m         new_future \u001b[38;5;241m=\u001b[39m asyncio\u001b[38;5;241m.\u001b[39mensure_future(result)\n",
+      "File \u001b[0;32m~/projects/llama_index/llama-index-core/llama_index/core/llms/callbacks.py:173\u001b[0m, in \u001b[0;36mllm_chat_callback.<locals>.wrap.<locals>.wrapped_llm_chat\u001b[0;34m(_self, messages, **kwargs)\u001b[0m\n\u001b[1;32m    164\u001b[0m event_id \u001b[38;5;241m=\u001b[39m callback_manager\u001b[38;5;241m.\u001b[39mon_event_start(\n\u001b[1;32m    165\u001b[0m     CBEventType\u001b[38;5;241m.\u001b[39mLLM,\n\u001b[1;32m    166\u001b[0m     payload\u001b[38;5;241m=\u001b[39m{\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    170\u001b[0m     },\n\u001b[1;32m    171\u001b[0m )\n\u001b[1;32m    172\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 173\u001b[0m     f_return_val \u001b[38;5;241m=\u001b[39m \u001b[43mf\u001b[49m\u001b[43m(\u001b[49m\u001b[43m_self\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    174\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    175\u001b[0m     callback_manager\u001b[38;5;241m.\u001b[39mon_event_end(\n\u001b[1;32m    176\u001b[0m         CBEventType\u001b[38;5;241m.\u001b[39mLLM,\n\u001b[1;32m    177\u001b[0m         payload\u001b[38;5;241m=\u001b[39m{EventPayload\u001b[38;5;241m.\u001b[39mEXCEPTION: e},\n\u001b[1;32m    178\u001b[0m         event_id\u001b[38;5;241m=\u001b[39mevent_id,\n\u001b[1;32m    179\u001b[0m     )\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/llama_index/llms/openai/base.py:379\u001b[0m, in \u001b[0;36mOpenAI.chat\u001b[0;34m(self, messages, **kwargs)\u001b[0m\n\u001b[1;32m    377\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    378\u001b[0m     chat_fn \u001b[38;5;241m=\u001b[39m completion_to_chat_decorator(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_complete)\n\u001b[0;32m--> 379\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mchat_fn\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/llama_index/llms/openai/base.py:107\u001b[0m, in \u001b[0;36mllm_retry_decorator.<locals>.wrapper\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m     98\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m f(\u001b[38;5;28mself\u001b[39m, \u001b[38;5;241m*\u001b[39margs, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs)\n\u001b[1;32m    100\u001b[0m retry \u001b[38;5;241m=\u001b[39m create_retry_decorator(\n\u001b[1;32m    101\u001b[0m     max_retries\u001b[38;5;241m=\u001b[39mmax_retries,\n\u001b[1;32m    102\u001b[0m     random_exponential\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    105\u001b[0m     max_seconds\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m20\u001b[39m,\n\u001b[1;32m    106\u001b[0m )\n\u001b[0;32m--> 107\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mretry\u001b[49m\u001b[43m(\u001b[49m\u001b[43mf\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/tenacity/__init__.py:336\u001b[0m, in \u001b[0;36mBaseRetrying.wraps.<locals>.wrapped_f\u001b[0;34m(*args, **kw)\u001b[0m\n\u001b[1;32m    334\u001b[0m copy \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mcopy()\n\u001b[1;32m    335\u001b[0m wrapped_f\u001b[38;5;241m.\u001b[39mstatistics \u001b[38;5;241m=\u001b[39m copy\u001b[38;5;241m.\u001b[39mstatistics  \u001b[38;5;66;03m# type: ignore[attr-defined]\u001b[39;00m\n\u001b[0;32m--> 336\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mcopy\u001b[49m\u001b[43m(\u001b[49m\u001b[43mf\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkw\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/tenacity/__init__.py:475\u001b[0m, in \u001b[0;36mRetrying.__call__\u001b[0;34m(self, fn, *args, **kwargs)\u001b[0m\n\u001b[1;32m    473\u001b[0m retry_state \u001b[38;5;241m=\u001b[39m RetryCallState(retry_object\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m, fn\u001b[38;5;241m=\u001b[39mfn, args\u001b[38;5;241m=\u001b[39margs, kwargs\u001b[38;5;241m=\u001b[39mkwargs)\n\u001b[1;32m    474\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m \u001b[38;5;28;01mTrue\u001b[39;00m:\n\u001b[0;32m--> 475\u001b[0m     do \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43miter\u001b[49m\u001b[43m(\u001b[49m\u001b[43mretry_state\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mretry_state\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    476\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(do, DoAttempt):\n\u001b[1;32m    477\u001b[0m         \u001b[38;5;28;01mtry\u001b[39;00m:\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/tenacity/__init__.py:376\u001b[0m, in \u001b[0;36mBaseRetrying.iter\u001b[0;34m(self, retry_state)\u001b[0m\n\u001b[1;32m    374\u001b[0m result \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    375\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m action \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39miter_state\u001b[38;5;241m.\u001b[39mactions:\n\u001b[0;32m--> 376\u001b[0m     result \u001b[38;5;241m=\u001b[39m \u001b[43maction\u001b[49m\u001b[43m(\u001b[49m\u001b[43mretry_state\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    377\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m result\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/tenacity/__init__.py:398\u001b[0m, in \u001b[0;36mBaseRetrying._post_retry_check_actions.<locals>.<lambda>\u001b[0;34m(rs)\u001b[0m\n\u001b[1;32m    396\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21m_post_retry_check_actions\u001b[39m(\u001b[38;5;28mself\u001b[39m, retry_state: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mRetryCallState\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m    397\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39miter_state\u001b[38;5;241m.\u001b[39mis_explicit_retry \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39miter_state\u001b[38;5;241m.\u001b[39mretry_run_result):\n\u001b[0;32m--> 398\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_add_action_func(\u001b[38;5;28;01mlambda\u001b[39;00m rs: \u001b[43mrs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43moutcome\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mresult\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m)\n\u001b[1;32m    399\u001b[0m         \u001b[38;5;28;01mreturn\u001b[39;00m\n\u001b[1;32m    401\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mafter \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n",
+      "File \u001b[0;32m~/.pyenv/versions/3.10.16/lib/python3.10/concurrent/futures/_base.py:451\u001b[0m, in \u001b[0;36mFuture.result\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    449\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m CancelledError()\n\u001b[1;32m    450\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_state \u001b[38;5;241m==\u001b[39m FINISHED:\n\u001b[0;32m--> 451\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m__get_result\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    453\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_condition\u001b[38;5;241m.\u001b[39mwait(timeout)\n\u001b[1;32m    455\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_state \u001b[38;5;129;01min\u001b[39;00m [CANCELLED, CANCELLED_AND_NOTIFIED]:\n",
+      "File \u001b[0;32m~/.pyenv/versions/3.10.16/lib/python3.10/concurrent/futures/_base.py:403\u001b[0m, in \u001b[0;36mFuture.__get_result\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    401\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_exception:\n\u001b[1;32m    402\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 403\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_exception\n\u001b[1;32m    404\u001b[0m     \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    405\u001b[0m         \u001b[38;5;66;03m# Break a reference cycle with the exception in self._exception\u001b[39;00m\n\u001b[1;32m    406\u001b[0m         \u001b[38;5;28mself\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/tenacity/__init__.py:478\u001b[0m, in \u001b[0;36mRetrying.__call__\u001b[0;34m(self, fn, *args, **kwargs)\u001b[0m\n\u001b[1;32m    476\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(do, DoAttempt):\n\u001b[1;32m    477\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 478\u001b[0m         result \u001b[38;5;241m=\u001b[39m \u001b[43mfn\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    479\u001b[0m     \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mBaseException\u001b[39;00m:  \u001b[38;5;66;03m# noqa: B902\u001b[39;00m\n\u001b[1;32m    480\u001b[0m         retry_state\u001b[38;5;241m.\u001b[39mset_exception(sys\u001b[38;5;241m.\u001b[39mexc_info())  \u001b[38;5;66;03m# type: ignore[arg-type]\u001b[39;00m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/llama_index/llms/openai/base.py:475\u001b[0m, in \u001b[0;36mOpenAI._chat\u001b[0;34m(self, messages, **kwargs)\u001b[0m\n\u001b[1;32m    469\u001b[0m message_dicts \u001b[38;5;241m=\u001b[39m to_openai_message_dicts(\n\u001b[1;32m    470\u001b[0m     messages,\n\u001b[1;32m    471\u001b[0m     model\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmodel,\n\u001b[1;32m    472\u001b[0m )\n\u001b[1;32m    474\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mreuse_client:\n\u001b[0;32m--> 475\u001b[0m     response \u001b[38;5;241m=\u001b[39m \u001b[43mclient\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mchat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcompletions\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mcreate\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    476\u001b[0m \u001b[43m        \u001b[49m\u001b[43mmessages\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmessage_dicts\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    477\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m    478\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_get_model_kwargs\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    479\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    480\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    481\u001b[0m     \u001b[38;5;28;01mwith\u001b[39;00m client:\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/openai/_utils/_utils.py:279\u001b[0m, in \u001b[0;36mrequired_args.<locals>.inner.<locals>.wrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    277\u001b[0m             msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMissing required argument: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mquote(missing[\u001b[38;5;241m0\u001b[39m])\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    278\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(msg)\n\u001b[0;32m--> 279\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfunc\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/openai/resources/chat/completions/completions.py:879\u001b[0m, in \u001b[0;36mCompletions.create\u001b[0;34m(self, messages, model, audio, frequency_penalty, function_call, functions, logit_bias, logprobs, max_completion_tokens, max_tokens, metadata, modalities, n, parallel_tool_calls, prediction, presence_penalty, reasoning_effort, response_format, seed, service_tier, stop, store, stream, stream_options, temperature, tool_choice, tools, top_logprobs, top_p, user, extra_headers, extra_query, extra_body, timeout)\u001b[0m\n\u001b[1;32m    837\u001b[0m \u001b[38;5;129m@required_args\u001b[39m([\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m], [\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmessages\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmodel\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mstream\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m    838\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mcreate\u001b[39m(\n\u001b[1;32m    839\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    876\u001b[0m     timeout: \u001b[38;5;28mfloat\u001b[39m \u001b[38;5;241m|\u001b[39m httpx\u001b[38;5;241m.\u001b[39mTimeout \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m|\u001b[39m NotGiven \u001b[38;5;241m=\u001b[39m NOT_GIVEN,\n\u001b[1;32m    877\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ChatCompletion \u001b[38;5;241m|\u001b[39m Stream[ChatCompletionChunk]:\n\u001b[1;32m    878\u001b[0m     validate_response_format(response_format)\n\u001b[0;32m--> 879\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_post\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    880\u001b[0m \u001b[43m        \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m/chat/completions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m,\u001b[49m\n\u001b[1;32m    881\u001b[0m \u001b[43m        \u001b[49m\u001b[43mbody\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmaybe_transform\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    882\u001b[0m \u001b[43m            \u001b[49m\u001b[43m{\u001b[49m\n\u001b[1;32m    883\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmessages\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    884\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodel\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    885\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43maudio\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43maudio\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    886\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfrequency_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfrequency_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    887\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunction_call\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunction_call\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    888\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mfunctions\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfunctions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    889\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogit_bias\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogit_bias\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    890\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mlogprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mlogprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    891\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_completion_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_completion_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    892\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmax_tokens\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmax_tokens\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    893\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmetadata\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmetadata\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    894\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmodalities\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodalities\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    895\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mn\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mn\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    896\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mparallel_tool_calls\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mparallel_tool_calls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    897\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mprediction\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mprediction\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    898\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mpresence_penalty\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpresence_penalty\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    899\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mreasoning_effort\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mreasoning_effort\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    900\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mresponse_format\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mresponse_format\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    901\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mseed\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mseed\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    902\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mservice_tier\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mservice_tier\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    903\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstop\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstop\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    904\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstore\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstore\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    905\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    906\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mstream_options\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_options\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    907\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtemperature\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtemperature\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    908\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtool_choice\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtool_choice\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    909\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtools\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    910\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_logprobs\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_logprobs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    911\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mtop_p\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mtop_p\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    912\u001b[0m \u001b[43m                \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43muser\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43muser\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    913\u001b[0m \u001b[43m            \u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    914\u001b[0m \u001b[43m            \u001b[49m\u001b[43mcompletion_create_params\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCompletionCreateParams\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    915\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    916\u001b[0m \u001b[43m        \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mmake_request_options\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    917\u001b[0m \u001b[43m            \u001b[49m\u001b[43mextra_headers\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_headers\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_query\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_query\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mextra_body\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mextra_body\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtimeout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mtimeout\u001b[49m\n\u001b[1;32m    918\u001b[0m \u001b[43m        \u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    919\u001b[0m \u001b[43m        \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mChatCompletion\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    920\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01mor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[1;32m    921\u001b[0m \u001b[43m        \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mStream\u001b[49m\u001b[43m[\u001b[49m\u001b[43mChatCompletionChunk\u001b[49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    922\u001b[0m \u001b[43m    \u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/openai/_base_client.py:1296\u001b[0m, in \u001b[0;36mSyncAPIClient.post\u001b[0;34m(self, path, cast_to, body, options, files, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1282\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21mpost\u001b[39m(\n\u001b[1;32m   1283\u001b[0m     \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m   1284\u001b[0m     path: \u001b[38;5;28mstr\u001b[39m,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1291\u001b[0m     stream_cls: \u001b[38;5;28mtype\u001b[39m[_StreamT] \u001b[38;5;241m|\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[1;32m   1292\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m ResponseT \u001b[38;5;241m|\u001b[39m _StreamT:\n\u001b[1;32m   1293\u001b[0m     opts \u001b[38;5;241m=\u001b[39m FinalRequestOptions\u001b[38;5;241m.\u001b[39mconstruct(\n\u001b[1;32m   1294\u001b[0m         method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpost\u001b[39m\u001b[38;5;124m\"\u001b[39m, url\u001b[38;5;241m=\u001b[39mpath, json_data\u001b[38;5;241m=\u001b[39mbody, files\u001b[38;5;241m=\u001b[39mto_httpx_files(files), \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions\n\u001b[1;32m   1295\u001b[0m     )\n\u001b[0;32m-> 1296\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m cast(ResponseT, \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequest\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mopts\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m)\u001b[49m)\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/openai/_base_client.py:973\u001b[0m, in \u001b[0;36mSyncAPIClient.request\u001b[0;34m(self, cast_to, options, remaining_retries, stream, stream_cls)\u001b[0m\n\u001b[1;32m    970\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m    971\u001b[0m     retries_taken \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m0\u001b[39m\n\u001b[0;32m--> 973\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    974\u001b[0m \u001b[43m    \u001b[49m\u001b[43mcast_to\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mcast_to\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    975\u001b[0m \u001b[43m    \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    976\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    977\u001b[0m \u001b[43m    \u001b[49m\u001b[43mstream_cls\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstream_cls\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    978\u001b[0m \u001b[43m    \u001b[49m\u001b[43mretries_taken\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mretries_taken\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    979\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/Library/Caches/pypoetry/virtualenvs/llama-index-VCjo73HL-py3.10/lib/python3.10/site-packages/openai/_base_client.py:1077\u001b[0m, in \u001b[0;36mSyncAPIClient._request\u001b[0;34m(self, cast_to, options, retries_taken, stream, stream_cls)\u001b[0m\n\u001b[1;32m   1074\u001b[0m         err\u001b[38;5;241m.\u001b[39mresponse\u001b[38;5;241m.\u001b[39mread()\n\u001b[1;32m   1076\u001b[0m     log\u001b[38;5;241m.\u001b[39mdebug(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mRe-raising status error\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m-> 1077\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_make_status_error_from_response(err\u001b[38;5;241m.\u001b[39mresponse) \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m   1079\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_process_response(\n\u001b[1;32m   1080\u001b[0m     cast_to\u001b[38;5;241m=\u001b[39mcast_to,\n\u001b[1;32m   1081\u001b[0m     options\u001b[38;5;241m=\u001b[39moptions,\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m   1085\u001b[0m     retries_taken\u001b[38;5;241m=\u001b[39mretries_taken,\n\u001b[1;32m   1086\u001b[0m )\n",
+      "\u001b[0;31mAuthenticationError\u001b[0m: Error code: 401 - {'error': {'message': 'Incorrect API key provided: key-KCj3*****************************************izwo. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}"
+     ]
+    }
+   ],
+   "source": [
+    "from llama_index.core.chat_engine.types import ChatMessage\n",
+    "\n",
+    "llm.chat([ChatMessage(role=\"user\", content=\"what color is the sky?\")])"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "llama-index-VCjo73HL-py3.10",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/pyproject.toml b/llama-index-integrations/llms/llama-index-llms-contextual/pyproject.toml
new file mode 100644
index 00000000000..9b6260befe3
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/pyproject.toml
@@ -0,0 +1,46 @@
+[build-system]
+build-backend = "poetry.core.masonry.api"
+requires = ["poetry-core"]
+
+[tool.codespell]
+check-filenames = true
+check-hidden = true
+skip = "*.csv,*.html,*.json,*.jsonl,*.pdf,*.txt,*.ipynb"
+
+[tool.llamahub]
+contains_example = false
+import_path = "llama_index.llms.contextual"
+
+[tool.llamahub.class_authors]
+Contextual = "sean-smith"
+
+[tool.mypy]
+disallow_untyped_defs = true
+exclude = ["_static", "build", "examples", "notebooks", "venv"]
+ignore_missing_imports = true
+python_version = "3.8"
+
+[tool.poetry]
+authors = ["Sean Smith <sean.smith@contextual.ai>"]
+description = "llama-index contextual integration"
+exclude = ["**/BUILD"]
+license = "MIT"
+name = "llama-index-llms-contextual"
+readme = "README.md"
+version = "0.0.1"
+
+[tool.poetry.dependencies]
+python = ">=3.9,<4.0"
+llama-index-llms-openai-like = "^0.3.3"
+contextual-client = "^0.4.0"
+
+[tool.poetry.group.dev.dependencies.black]
+extras = ["jupyter"]
+version = "<=23.9.1,>=23.7.0"
+
+[tool.poetry.group.dev.dependencies.codespell]
+extras = ["toml"]
+version = ">=v2.2.6"
+
+[[tool.poetry.packages]]
+include = "llama_index/"
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/tests/BUILD b/llama-index-integrations/llms/llama-index-llms-contextual/tests/BUILD
new file mode 100644
index 00000000000..db46e8d6c97
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/tests/BUILD
@@ -0,0 +1 @@
+python_sources()
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/tests/__init__.py b/llama-index-integrations/llms/llama-index-llms-contextual/tests/__init__.py
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py b/llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py
new file mode 100644
index 00000000000..b27c650dce8
--- /dev/null
+++ b/llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py
@@ -0,0 +1,7 @@
+from llama_index.core.base.llms.base import BaseLLM
+from llama_index.llms.contextual import Contextual
+
+
+def test_llm_class():
+    names_of_base_classes = [b.__name__ for b in Contextual.__mro__]
+    assert BaseLLM.__name__ in names_of_base_classes


Reviews:
user: login: logan-markewich
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-28T21:49:42Z
user: login: logan-markewich
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-02-28T21:50:46Z
user: login: sean-smith
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-02-28T22:44:40Z
user: login: sean-smith
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T22:26:54Z
user: login: sean-smith
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-04T01:53:42Z
user: login: logan-markewich
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T00:17:09Z
user: login: sean-smith
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T03:40:17Z
user: login: logan-markewich
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-03-12T09:53:30Z

Review comments:
path: llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py
user: login: logan-markewich
body: This is not a real unit test (also, not sure if you meant to leak this key or not)
created_at: 2025-02-28T21:49:42Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,117 @@
+import os
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    CompletionResponseGen,
+    ChatResponse,
+    ChatResponseGen,
+    ChatResponseAsyncGen,
+    CompletionResponseAsyncGen,
+    LLMMetadata,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(description="The model to use. Currently only supports `v1`.")
+    api_key: str = Field(description="The API key to use.", default=os.environ.get("API_KEY", None))
+    base_url: str = Field(description="The base URL to use.", default="https://api.contextual.com")
+    system_prompt: str = Field(description="Instructions that the model follows when generating responses. Note that we do not guarantee that the model follows these instructions exactly.")
+    avoid_commentary: bool = Field(description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.", default=False)
+    client: Any = Field(default=None, exclude=True, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: Optional[str] = None,
+        base_url: str = "https://api.contextual.com",
+        system_prompt: str = "",
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        api_key = api_key or os.environ.get("API_KEY", None)
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Failed to initialize Contextual client: {e}")
+
+
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+    
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(self, prompt: str, **kwargs) -> CompletionResponse:
path: llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
user: login: logan-markewich
body: We probably want to implement more methods here?

although if the API endpoint is openai-compatible, all we need to do is implement the proper api_base and other init args, and the rest will work
created_at: 2025-02-28T21:50:46Z
author_association: COLLABORATOR
path: llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py
user: login: sean-smith
body: @logan-markewich yeah the key is invalidated but you're right I should remove this from the unit tests.
created_at: 2025-02-28T22:44:40Z
author_association: CONTRIBUTOR
path: llama-index-integrations/llms/llama-index-llms-contextual/tests/test.py
user: login: sean-smith
body: Removed
created_at: 2025-03-03T22:26:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,117 @@
+import os
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    CompletionResponseGen,
+    ChatResponse,
+    ChatResponseGen,
+    ChatResponseAsyncGen,
+    CompletionResponseAsyncGen,
+    LLMMetadata,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(description="The model to use. Currently only supports `v1`.")
+    api_key: str = Field(description="The API key to use.", default=os.environ.get("API_KEY", None))
+    base_url: str = Field(description="The base URL to use.", default="https://api.contextual.com")
+    system_prompt: str = Field(description="Instructions that the model follows when generating responses. Note that we do not guarantee that the model follows these instructions exactly.")
+    avoid_commentary: bool = Field(description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.", default=False)
+    client: Any = Field(default=None, exclude=True, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: Optional[str] = None,
+        base_url: str = "https://api.contextual.com",
+        system_prompt: str = "",
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        api_key = api_key or os.environ.get("API_KEY", None)
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Failed to initialize Contextual client: {e}")
+
+
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+    
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(self, prompt: str, **kwargs) -> CompletionResponse:
path: llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
user: login: sean-smith
body: @logan-markewich I've implemented the `chat` and `complete` interfaces. We don't support streaming or async (yet) so that's all users can use right now.
created_at: 2025-03-04T01:53:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,158 @@
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from typing import List
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    ChatResponse,
+    ChatResponseGen,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(
+        description="The model to use. Currently only supports `v1`.", default="v1"
+    )
+    api_key: str = Field(description="The API key to use.", default=None)
+    base_url: str = Field(
+        description="The base URL to use.",
+        default="https://api.contextual.ai/v1/generate",
+    )
+    avoid_commentary: bool = Field(
+        description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.",
+        default=False,
+    )
+    client: Any = Field(default=None, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str,
+        base_url: str = None,
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Error initializing ContextualAI client: {e}")
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(
+        self, prompt: str, knowledge: Optional[List[str]] = None, **kwargs
+    ) -> CompletionResponse:
+        """
+        Generate completion for the given prompt.
+
+        Args:
+            prompt (str): The input prompt to generate completion for.
+            **kwargs: Additional keyword arguments for the API request.
+
+        Returns:
+            str: The generated text completion.
+        """
+        messages_list = [{"role": MessageRole.USER, "content": prompt}]
+        response = self._generate(
+            knowledge=knowledge,
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return CompletionResponse(text=response)
+
+    @llm_chat_callback()
+    def chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponse:
+        """
+        Generate a chat response for the given messages.
+        """
+        messages_list = [
+            {"role": msg.role, "content": msg.blocks[0].text} for msg in messages
+        ]
+        response = self._generate(
+            knowledge=kwargs.get("knowledge_base", None),
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return ChatResponse(
+            message=ChatMessage(role=MessageRole.ASSISTANT, content=response)
+        )
+
+    @llm_chat_callback()
+    def stream_chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    def stream_complete(self, prompt: str, **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    def _generate(
path: llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
user: login: logan-markewich
body: This seems fairly open ai compatible -- any reason not not just use the base openai client and change the model args?

Also what you have here doesn't override any async methods, so those will still call the normal openai client 
created_at: 2025-03-12T00:17:09Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,158 @@
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from typing import List
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    ChatResponse,
+    ChatResponseGen,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(
+        description="The model to use. Currently only supports `v1`.", default="v1"
+    )
+    api_key: str = Field(description="The API key to use.", default=None)
+    base_url: str = Field(
+        description="The base URL to use.",
+        default="https://api.contextual.ai/v1/generate",
+    )
+    avoid_commentary: bool = Field(
+        description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.",
+        default=False,
+    )
+    client: Any = Field(default=None, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str,
+        base_url: str = None,
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Error initializing ContextualAI client: {e}")
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(
+        self, prompt: str, knowledge: Optional[List[str]] = None, **kwargs
+    ) -> CompletionResponse:
+        """
+        Generate completion for the given prompt.
+
+        Args:
+            prompt (str): The input prompt to generate completion for.
+            **kwargs: Additional keyword arguments for the API request.
+
+        Returns:
+            str: The generated text completion.
+        """
+        messages_list = [{"role": MessageRole.USER, "content": prompt}]
+        response = self._generate(
+            knowledge=knowledge,
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return CompletionResponse(text=response)
+
+    @llm_chat_callback()
+    def chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponse:
+        """
+        Generate a chat response for the given messages.
+        """
+        messages_list = [
+            {"role": msg.role, "content": msg.blocks[0].text} for msg in messages
+        ]
+        response = self._generate(
+            knowledge=kwargs.get("knowledge_base", None),
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return ChatResponse(
+            message=ChatMessage(role=MessageRole.ASSISTANT, content=response)
+        )
+
+    @llm_chat_callback()
+    def stream_chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    def stream_complete(self, prompt: str, **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    def _generate(
path: llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
user: login: sean-smith
body: @logan-markewich The issue here is we're not 100% openai compatible. We structure our URL's differently, i.e. our api is called like so:

```python
import requests

url = "https://api.contextual.ai/v1/generate" # <- this isn't openai compatible since we don't have /chat/completions

payload = {
    "messages": [{ "role": "user" }],
    "avoid_commentary": False,
    "temperature": 0,
    "top_p": 0.9,
    "max_new_tokens": 1024
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)
```

I've added the `NotImplementedError` for the async methods. Thanks for calling that out!



created_at: 2025-03-12T03:40:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,158 @@
+from typing import Any, Optional
+
+from llama_index.llms.openai_like import OpenAILike
+from pydantic import Field
+from typing import List
+from llama_index.core.llms.callbacks import (
+    llm_chat_callback,
+    llm_completion_callback,
+)
+from llama_index.core.base.llms.types import (
+    CompletionResponse,
+    ChatResponse,
+    ChatResponseGen,
+    MessageRole,
+    ChatMessage,
+)
+
+
+from contextual import ContextualAI
+
+
+class Contextual(OpenAILike):
+    """
+    Generate a response using Contextual's Grounded Language Model (GLM), an LLM engineered specifically to prioritize faithfulness to in-context retrievals over parametric knowledge to reduce hallucinations in Retrieval-Augmented Generation.
+
+    The total request cannot exceed 32,000 tokens. Email glm-feedback@contextual.ai with any feedback or questions.
+
+    Examples:
+        `pip install llama-index-llms-contextual`
+
+        ```python
+        from llama_index.llms.contextual import Contextual
+
+        # Set up the Contextual class with the required model and API key
+        llm = Contextual(model="contextual-clm", api_key="your_api_key")
+
+        # Call the complete method with a query
+        response = llm.complete("Explain the importance of low latency LLMs")
+
+        print(response)
+        ```
+    """
+
+    model: str = Field(
+        description="The model to use. Currently only supports `v1`.", default="v1"
+    )
+    api_key: str = Field(description="The API key to use.", default=None)
+    base_url: str = Field(
+        description="The base URL to use.",
+        default="https://api.contextual.ai/v1/generate",
+    )
+    avoid_commentary: bool = Field(
+        description="Flag to indicate whether the model should avoid providing additional commentary in responses. Commentary is conversational in nature and does not contain verifiable claims; therefore, commentary is not strictly grounded in available context. However, commentary may provide useful context which improves the helpfulness of responses.",
+        default=False,
+    )
+    client: Any = Field(default=None, description="Contextual AI Client")
+
+    def __init__(
+        self,
+        model: str,
+        api_key: str,
+        base_url: str = None,
+        avoid_commentary: bool = False,
+        **openai_llm_kwargs: Any,
+    ) -> None:
+        super().__init__(
+            model=model,
+            api_key=api_key,
+            api_base=base_url,
+            is_chat_model=openai_llm_kwargs.pop("is_chat_model", True),
+            **openai_llm_kwargs,
+        )
+
+        try:
+            self.client = ContextualAI(api_key=api_key, base_url=base_url)
+        except Exception as e:
+            raise ValueError(f"Error initializing ContextualAI client: {e}")
+
+    @classmethod
+    def class_name(cls) -> str:
+        """Get class name."""
+        return "contextual-clm"
+
+    # Synchronous Methods
+    @llm_completion_callback()
+    def complete(
+        self, prompt: str, knowledge: Optional[List[str]] = None, **kwargs
+    ) -> CompletionResponse:
+        """
+        Generate completion for the given prompt.
+
+        Args:
+            prompt (str): The input prompt to generate completion for.
+            **kwargs: Additional keyword arguments for the API request.
+
+        Returns:
+            str: The generated text completion.
+        """
+        messages_list = [{"role": MessageRole.USER, "content": prompt}]
+        response = self._generate(
+            knowledge=knowledge,
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return CompletionResponse(text=response)
+
+    @llm_chat_callback()
+    def chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponse:
+        """
+        Generate a chat response for the given messages.
+        """
+        messages_list = [
+            {"role": msg.role, "content": msg.blocks[0].text} for msg in messages
+        ]
+        response = self._generate(
+            knowledge=kwargs.get("knowledge_base", None),
+            messages=messages_list,
+            model=self.model,
+            system_prompt=self.system_prompt,
+            **kwargs,
+        )
+        return ChatResponse(
+            message=ChatMessage(role=MessageRole.ASSISTANT, content=response)
+        )
+
+    @llm_chat_callback()
+    def stream_chat(self, messages: List[ChatMessage], **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    @llm_completion_callback()
+    def stream_complete(self, prompt: str, **kwargs) -> ChatResponseGen:
+        """
+        Generate a chat response for the given messages.
+        """
+        raise NotImplementedError("stream methods not implemented in Contextual")
+
+    def _generate(
path: llama-index-integrations/llms/llama-index-llms-contextual/llama_index/llms/contextual/base.py
user: login: logan-markewich
body: gotcha
created_at: 2025-03-12T09:53:30Z
author_association: COLLABORATOR

PR comments:
user: login: review-notebook-app[bot]
created_at: 2025-02-28T01:13:38Z
body: Check out this pull request on&nbsp; <a href="https://app.reviewnb.com/run-llama/llama_index/pull/17913"><img align="absmiddle"  alt="ReviewNB" height="28" class="BotMessageButtonImage" src="https://raw.githubusercontent.com/ReviewNB/support/master/images/button_reviewnb.png"/></a> 

 See visual diffs & provide feedback on Jupyter Notebooks. 

---

 <i>Powered by <a href='https://www.reviewnb.com/?utm_source=gh'>ReviewNB</a></i>
author_association: NONE
user: login: sean-smith
created_at: 2025-03-04T01:54:39Z
body: @logan-markewich can you approve the workflows?
author_association: CONTRIBUTOR
user: login: sean-smith
created_at: 2025-03-07T00:01:44Z
body: Pinging on this @logan-markewich 
author_association: CONTRIBUTOR