Loading Indices

llama_index.indices.loading.load_graph_from_storage(storage_context: StorageContext, root_id: str, **kwargs: Any) -> ComposableGraph
   Load composable graph from storage context.
   Parameters:
      * **storage_context** (*StorageContext*) -- storage context
        containing docstore, index store and vector store.
      * **root_id** (*str*) -- ID of the root index of the graph.
      * ****kwargs** -- Additional keyword args to pass to the index
        constructors.
llama_index.indices.loading.load_index_from_storage(storage_context: StorageContext, index_id: Optional[str] = None, **kwargs: Any) -> BaseIndex
   Load index from storage context.
   Parameters:
      * **storage_context** (*StorageContext*) -- storage context
        containing docstore, index store and vector store.
      * **index_id** (*Optional**[**str**]*) -- ID of the index to
        load. Defaults to None, which assumes there's only a single
        index in the index store and load it.
      * ****kwargs** -- Additional keyword args to pass to the index
        constructors.
llama_index.indices.loading.load_indices_from_storage(storage_context: StorageContext, index_ids: Optional[Sequence[str]] = None, **kwargs: Any) -> List[BaseIndex]
   Load multiple indices from storage context.
   Parameters:
      * **storage_context** (*StorageContext*) -- storage context
        containing docstore, index store and vector store.
      * **index_id** (*Optional**[**Sequence**[**str**]**]*) -- IDs of
        the indices to load. Defaults to None, which loads all indices
        in the index store.
      * ****kwargs** -- Additional keyword args to pass to the index
        constructors.


KV Storage

class llama_index.storage.kvstore.FirestoreKVStore(project: Optional[str] = None, database: str = '(default)')
   Firestore Key-Value store.
   Parameters:
      * **project** (*str*) -- The project which the client acts on
        behalf of.
      * **database** (*str*) -- The database name that the client
        targets.
   delete(key: str, collection: str = 'data') -> bool
      Delete a value from the Firestore.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   get(key: str, collection: str = 'data') -> Optional[dict]
      Get a key-value pair from the Firestore.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   get_all(collection: str = 'data') -> Dict[str, dict]
      Get all values from the Firestore collection.
      Parameters:
         **collection** (*str*) -- collection name
   put(key: str, val: dict, collection: str = 'data') -> None
      Put a key-value pair into the Firestore collection.
      Parameters:
         * **key** (*str*) -- key
         * **val** (*dict*) -- value
         * **collection** (*str*) -- collection name
class llama_index.storage.kvstore.MongoDBKVStore(mongo_client: Any, uri: Optional[str] = None, host: Optional[str] = None, port: Optional[int] = None, db_name: Optional[str] = None)
   MongoDB Key-Value store.
   Parameters:
      * **mongo_client** (*Any*) -- MongoDB client
      * **uri** (*Optional**[**str**]*) -- MongoDB URI
      * **host** (*Optional**[**str**]*) -- MongoDB host
      * **port** (*Optional**[**int**]*) -- MongoDB port
      * **db_name** (*Optional**[**str**]*) -- MongoDB database name
   delete(key: str, collection: str = 'data') -> bool
      Delete a value from the store.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   classmethod from_host_and_port(host: str, port: int, db_name: Optional[str] = None) -> MongoDBKVStore
      Load a MongoDBKVStore from a MongoDB host and port.
      Parameters:
         * **host** (*str*) -- MongoDB host
         * **port** (*int*) -- MongoDB port
         * **db_name** (*Optional**[**str**]*) -- MongoDB database
           name
   classmethod from_uri(uri: str, db_name: Optional[str] = None) -> MongoDBKVStore
      Load a MongoDBKVStore from a MongoDB URI.
      Parameters:
         * **uri** (*str*) -- MongoDB URI
         * **db_name** (*Optional**[**str**]*) -- MongoDB database
           name
   get(key: str, collection: str = 'data') -> Optional[dict]
      Get a value from the store.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   get_all(collection: str = 'data') -> Dict[str, dict]
      Get all values from the store.
      Parameters:
         **collection** (*str*) -- collection name
   put(key: str, val: dict, collection: str = 'data') -> None
      Put a key-value pair into the store.
      Parameters:
         * **key** (*str*) -- key
         * **val** (*dict*) -- value
         * **collection** (*str*) -- collection name
class llama_index.storage.kvstore.RedisKVStore(redis_uri: Optional[str] = 'redis://127.0.0.1:6379', **kwargs: Any)

KV Storage

   Redis KV Store.
   Parameters:
      * **redis_client** (*Any*) -- Redis client
      * **redis_url** (*Optional**[**str**]*) -- Redis server URI
   Raises:
      **ValueError** -- If redis-py is not installed
   -[ Examples ]-
   >>> from llama_index.storage.kvstore.redis_kvstore import RedisKVStore
   >>> # Create a RedisKVStore
   >>> redis_kv_store = RedisKVStore(
   >>>     redis_url="redis://127.0.0.1:6379")
   delete(key: str, collection: str = 'data') -> bool
      Delete a value from the store.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   classmethod from_host_and_port(host: str, port: int) -> RedisKVStore
      Load a RedisKVStore from a Redis host and port.
      Parameters:
         * **host** (*str*) -- Redis host
         * **port** (*int*) -- Redis port
   classmethod from_redis_client(redis_client: Any) -> RedisKVStore
      Load a RedisKVStore from a Redis Client.
      Parameters:
         **redis_client** (*Redis*) -- Redis client
   get(key: str, collection: str = 'data') -> Optional[dict]
      Get a value from the store.
      Parameters:
         * **key** (*str*) -- key
         * **collection** (*str*) -- collection name
   get_all(collection: str = 'data') -> Dict[str, dict]
      Get all values from the store.
   put(key: str, val: dict, collection: str = 'data') -> None
      Put a key-value pair into the store.
      Parameters:
         * **key** (*str*) -- key
         * **val** (*dict*) -- value
         * **collection** (*str*) -- collection name
class llama_index.storage.kvstore.SimpleKVStore(data: Optional[Dict[str, Dict[str, dict]]] = None)
   Simple in-memory Key-Value store.
   Parameters:
      **data** (*Optional**[**DATA_TYPE**]*) -- data to initialize the
      store with
   delete(key: str, collection: str = 'data') -> bool
      Delete a value from the store.
   classmethod from_dict(save_dict: dict) -> SimpleKVStore
      Load a SimpleKVStore from dict.
   classmethod from_persist_path(persist_path: str, fs: Optional[AbstractFileSystem] = None) -> SimpleKVStore
      Load a SimpleKVStore from a persist path and filesystem.
   get(key: str, collection: str = 'data') -> Optional[dict]
      Get a value from the store.
   get_all(collection: str = 'data') -> Dict[str, dict]
      Get all values from the store.
   persist(persist_path: str, fs: Optional[AbstractFileSystem] = None) -> None
      Persist the store.
   put(key: str, val: dict, collection: str = 'data') -> None
      Put a key-value pair into the store.
   to_dict() -> dict
      Save the store as dict.


HuggingFaceLLM

pydantic model llama_index.llms.huggingface.HuggingFaceLLM
   HuggingFace LLM.
      {
         "title": "HuggingFaceLLM",
         "description": "HuggingFace LLM.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model_name": {
               "title": "Model Name",
               "description": "The model name to use from HuggingFace. Unused if `model` is passed in directly.",
               "type": "string"
            },
            "context_window": {
               "title": "Context Window",
               "description": "The maximum number of tokens available for input.",
               "type": "integer"
            },
            "max_new_tokens": {
               "title": "Max New Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "system_prompt": {
               "title": "System Prompt",
               "description": "The system prompt, containing any extra instructions or context. The model card on HuggingFace should specify if this is needed.",
               "type": "string"
            },
            "query_wrapper_prompt": {
               "title": "Query Wrapper Prompt",
               "description": "The query wrapper prompt, containing the query placeholder. The model card on HuggingFace should specify if this is needed. Should contain a `{query_str}` placeholder.",
               "type": "string"
            },
            "tokenizer_name": {
               "title": "Tokenizer Name",
               "description": "The name of the tokenizer to use from HuggingFace. Unused if `tokenizer` is passed in directly.",
               "type": "string"
            },
            "device_map": {
               "title": "Device Map",
               "description": "The device_map to use. Defaults to 'auto'.",
               "type": "string"
            },
            "stopping_ids": {
               "title": "Stopping Ids",
               "description": "The stopping ids to use. Generation stops when these token IDs are predicted.",
               "type": "array",
               "items": {
                  "type": "integer"
               }
            },
            "tokenizer_outputs_to_remove": {
               "title": "Tokenizer Outputs To Remove",
               "description": "The outputs to remove from the tokenizer. Sometimes huggingface tokenizers return extra inputs that cause errors.",
               "type": "array",
               "items": {}
            },
            "tokenizer_kwargs": {
               "title": "Tokenizer Kwargs",
               "description": "The kwargs to pass to the tokenizer.",
               "type": "object"
            },
            "model_kwargs": {
               "title": "Model Kwargs",
               "description": "The kwargs to pass to the model during initialization.",
               "type": "object"
            },
            "generate_kwargs": {
               "title": "Generate Kwargs",
               "description": "The kwargs to pass to the model during generation.",
               "type": "object"
            }
         },
         "required": [
            "model_name",
            "context_window",
            "max_new_tokens",
            "system_prompt",
            "query_wrapper_prompt",
            "tokenizer_name"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "context_window (int)"
      * "device_map (Optional[str])"
      * "generate_kwargs (dict)"
      * "max_new_tokens (int)"
      * "model_kwargs (dict)"
      * "model_name (str)"
      * "query_wrapper_prompt (str)"
      * "stopping_ids (List[int])"
      * "system_prompt (str)"
      * "tokenizer_kwargs (dict)"
      * "tokenizer_name (str)"
      * "tokenizer_outputs_to_remove (list)"

HuggingFaceLLM

   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field context_window: int [Required]
      The maximum number of tokens available for input.
   field device_map: Optional[str] = None
      The device_map to use. Defaults to 'auto'.
   field generate_kwargs: dict [Optional]
      The kwargs to pass to the model during generation.
   field max_new_tokens: int [Required]
      The maximum number of tokens to generate.
   field model_kwargs: dict [Optional]
      The kwargs to pass to the model during initialization.
   field model_name: str [Required]
      The model name to use from HuggingFace. Unused if *model* is
      passed in directly.
   field query_wrapper_prompt: str [Required]
      The query wrapper prompt, containing the query placeholder. The
      model card on HuggingFace should specify if this is needed.
      Should contain a *{query_str}* placeholder.
   field stopping_ids: List[int] [Optional]
      The stopping ids to use. Generation stops when these token IDs
      are predicted.
   field system_prompt: str [Required]
      The system prompt, containing any extra instructions or context.
      The model card on HuggingFace should specify if this is needed.
   field tokenizer_kwargs: dict [Optional]
      The kwargs to pass to the tokenizer.
   field tokenizer_name: str [Required]
      The name of the tokenizer to use from HuggingFace. Unused if
      *tokenizer* is passed in directly.
   field tokenizer_outputs_to_remove: list [Optional]
      The outputs to remove from the tokenizer. Sometimes huggingface
      tokenizers return extra inputs that cause errors.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


LiteLLM

pydantic model llama_index.llms.litellm.LiteLLM
      {
         "title": "LiteLLM",
         "description": "LLM interface.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "The LiteLLM model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use during generation.",
               "type": "number"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "description": "Additional kwargs for the LLM API.",
               "type": "object"
            },
            "max_retries": {
               "title": "Max Retries",
               "description": "The maximum number of API retries.",
               "type": "integer"
            }
         },
         "required": [
            "model",
            "temperature",
            "max_retries"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "additional_kwargs (Dict[str, Any])"
      * "max_retries (int)"
      * "max_tokens (Optional[int])"
      * "model (str)"
      * "temperature (float)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field additional_kwargs: Dict[str, Any] [Optional]
      Additional kwargs for the LLM API.
   field max_retries: int [Required]
      The maximum number of API retries.
   field max_tokens: Optional[int] = None
      The maximum number of tokens to generate.
   field model: str [Required]
      The LiteLLM model to use.
   field temperature: float [Required]
      The temperature to use during generation.
   async achat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async chat endpoint for LLM.
   async acomplete(*args: Any, **kwargs: Any) -> Any
      Async completion endpoint for LLM.
   async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async streaming chat endpoint for LLM.
   async astream_complete(*args: Any, **kwargs: Any) -> Any
      Async streaming completion endpoint for LLM.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Azure OpenAI

pydantic model llama_index.llms.azure_openai.AzureOpenAI
   Azure OpenAI.
   To use this, you must first deploy a model on Azure OpenAI. Unlike
   OpenAI, you need to specify a *engine* parameter to identify your
   deployment (called "model deployment name" in Azure portal).
   * model: Name of the model (e.g. *text-davinci-003*)
        This in only used to decide completion vs. chat endpoint.
   * engine: This will correspond to the custom name you chose
        for your deployment when you deployed a model.
   You must have the following environment variables set: -
   *OPENAI_API_TYPE*: set this to *azure*, *azure_ad*, or *azuread* -
   *OPENAI_API_VERSION*: set this to *2023-05-15*
      This may change in the future.
   * *OPENAI_API_BASE*: your endpoint should look like the following
        https://YOUR_RESOURCE_NAME.openai.azure.com/
   * *OPENAI_API_KEY*: your API key
   More information can be found here:
      https://learn.microsoft.com/en-us/azure/cognitive-
      services/openai/quickstart?tabs=command-line&pivots=programming-
      language-python
      {
         "title": "AzureOpenAI",
         "description": "Azure OpenAI.\n\nTo use this, you must first deploy a model on Azure OpenAI.\nUnlike OpenAI, you need to specify a `engine` parameter to identify\nyour deployment (called \"model deployment name\" in Azure portal).\n\n- model: Name of the model (e.g. `text-davinci-003`)\n    This in only used to decide completion vs. chat endpoint.\n- engine: This will correspond to the custom name you chose\n    for your deployment when you deployed a model.\n\nYou must have the following environment variables set:\n- `OPENAI_API_TYPE`: set this to `azure`, `azure_ad`, or `azuread`\n- `OPENAI_API_VERSION`: set this to `2023-05-15`\n    This may change in the future.\n- `OPENAI_API_BASE`: your endpoint should look like the following\n    https://YOUR_RESOURCE_NAME.openai.azure.com/\n- `OPENAI_API_KEY`: your API key\n\nMore information can be found here:\n    https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?tabs=command-line&pivots=programming-language-python",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "The OpenAI model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use during generation.",
               "type": "number"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "description": "Additional kwargs for the OpenAI API.",
               "type": "object"
            },
            "max_retries": {
               "title": "Max Retries",
               "description": "The maximum number of API retries.",
               "type": "integer"
            },
            "api_key": {
               "title": "Api Key",
               "description": "The OpenAI API key.",
               "type": "string"
            },
            "api_type": {
               "title": "Api Type",
               "description": "The OpenAI API type.",
               "type": "string"

Azure OpenAI

            },
            "api_base": {
               "title": "Api Base",
               "description": "The base URL for OpenAI API.",
               "type": "string"
            },
            "api_version": {
               "title": "Api Version",
               "description": "The API version for OpenAI API.",
               "type": "string"
            },
            "engine": {
               "title": "Engine",
               "description": "The name of the deployed azure engine.",
               "type": "string"
            }
         },
         "required": [
            "model",
            "temperature",
            "max_retries",
            "api_base",
            "api_version",
            "engine"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "engine (str)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
      * "validate_env" » "all fields"
   field engine: str [Required]
      The name of the deployed azure engine.
      Validated by:
         * "validate_env"
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   validator validate_env  »  *all fields*
      Validate necessary credentials are set.


LangChainLLM

pydantic model llama_index.llms.langchain.LangChainLLM
   Adapter for a LangChain LLM.
      {
         "title": "LangChainLLM",
         "description": "Adapter for a LangChain LLM.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   async achat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async chat endpoint for LLM.
   async acomplete(*args: Any, **kwargs: Any) -> Any
      Async completion endpoint for LLM.
   async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async streaming chat endpoint for LLM.
   async astream_complete(*args: Any, **kwargs: Any) -> Any
      Async streaming completion endpoint for LLM.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property llm: BaseLanguageModel
   property metadata: LLMMetadata
      LLM metadata.


PaLM

pydantic model llama_index.llms.palm.PaLM
   PaLM LLM.
      {
         "title": "PaLM",
         "description": "PaLM LLM.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model_name": {
               "title": "Model Name",
               "description": "The PaLM model to use.",
               "type": "string"
            },
            "num_output": {
               "title": "Num Output",
               "description": "The number of tokens to generate.",
               "type": "integer"
            },
            "generate_kwargs": {
               "title": "Generate Kwargs",
               "description": "Kwargs for generation.",
               "type": "object"
            }
         },
         "required": [
            "model_name",
            "num_output"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "generate_kwargs (dict)"
      * "model_name (str)"
      * "num_output (int)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field generate_kwargs: dict [Optional]
      Kwargs for generation.
   field model_name: str [Required]
      The PaLM model to use.
   field num_output: int [Required]
      The number of tokens to generate.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      Get LLM metadata.


OpenAI

pydantic model llama_index.llms.openai.OpenAI
      {
         "title": "OpenAI",
         "description": "LLM interface.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "The OpenAI model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use during generation.",
               "type": "number"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "description": "Additional kwargs for the OpenAI API.",
               "type": "object"
            },
            "max_retries": {
               "title": "Max Retries",
               "description": "The maximum number of API retries.",
               "type": "integer"
            },
            "api_key": {
               "title": "Api Key",
               "description": "The OpenAI API key.",
               "type": "string"
            },
            "api_type": {
               "title": "Api Type",
               "description": "The OpenAI API type.",
               "type": "string"
            },
            "api_base": {
               "title": "Api Base",
               "description": "The base URL for OpenAI API.",
               "type": "string"
            },
            "api_version": {
               "title": "Api Version",
               "description": "The API version for OpenAI API.",
               "type": "string"
            }
         },
         "required": [
            "model",
            "temperature",
            "max_retries",
            "api_base",
            "api_version"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "additional_kwargs (Dict[str, Any])"
      * "api_base (str)"
      * "api_key (str)"
      * "api_type (str)"
      * "api_version (str)"
      * "max_retries (int)"
      * "max_tokens (Optional[int])"
      * "model (str)"
      * "temperature (float)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field additional_kwargs: Dict[str, Any] [Optional]
      Additional kwargs for the OpenAI API.
   field api_base: str [Required]
      The base URL for OpenAI API.
   field api_key: str = None
      The OpenAI API key.
   field api_type: str = None
      The OpenAI API type.
   field api_version: str [Required]
      The API version for OpenAI API.
   field max_retries: int [Required]
      The maximum number of API retries.
   field max_tokens: Optional[int] = None
      The maximum number of tokens to generate.
   field model: str [Required]
      The OpenAI model to use.
   field temperature: float [Required]
      The temperature to use during generation.
   async achat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async chat endpoint for LLM.
   async acomplete(*args: Any, **kwargs: Any) -> Any
      Async completion endpoint for LLM.
   async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async streaming chat endpoint for LLM.
   async astream_complete(*args: Any, **kwargs: Any) -> Any
      Async streaming completion endpoint for LLM.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any

OpenAI

      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Anthropic

pydantic model llama_index.llms.anthropic.Anthropic
      {
         "title": "Anthropic",
         "description": "LLM interface.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "The anthropic model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use for sampling.",
               "type": "number"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "base_url": {
               "title": "Base Url",
               "description": "The base URL to use.",
               "type": "string"
            },
            "timeout": {
               "title": "Timeout",
               "description": "The timeout to use in seconds.",
               "type": "number"
            },
            "max_retries": {
               "title": "Max Retries",
               "description": "The maximum number of API retries.",
               "default": 10,
               "type": "integer"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "description": "Additional kwargs for the anthropic API.",
               "type": "object"
            }
         },
         "required": [
            "model",
            "temperature",
            "max_tokens"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "additional_kwargs (Dict[str, Any])"
      * "base_url (Optional[str])"
      * "max_retries (int)"
      * "max_tokens (int)"
      * "model (str)"
      * "temperature (float)"
      * "timeout (Optional[float])"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field additional_kwargs: Dict[str, Any] [Optional]
      Additional kwargs for the anthropic API.
   field base_url: Optional[str] = None
      The base URL to use.
   field max_retries: int = 10
      The maximum number of API retries.
   field max_tokens: int [Required]
      The maximum number of tokens to generate.
   field model: str [Required]
      The anthropic model to use.
   field temperature: float [Required]
      The temperature to use for sampling.
   field timeout: Optional[float] = None
      The timeout to use in seconds.
   async achat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async chat endpoint for LLM.
   async acomplete(*args: Any, **kwargs: Any) -> Any
      Async completion endpoint for LLM.
   async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Async streaming chat endpoint for LLM.
   async astream_complete(*args: Any, **kwargs: Any) -> Any
      Async streaming completion endpoint for LLM.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.

Anthropic

   property metadata: LLMMetadata
      LLM metadata.


XOrbits Xinference

pydantic model llama_index.llms.xinference.Xinference
      {
         "title": "Xinference",
         "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `complete`,\n    `stream_complete`, and `metadata` methods.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model_uid": {
               "title": "Model Uid",
               "description": "The Xinference model to use.",
               "type": "string"
            },
            "endpoint": {
               "title": "Endpoint",
               "description": "The Xinference endpoint URL to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use for sampling.",
               "type": "number"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The maximum new tokens to generate as answer.",
               "type": "integer"
            },
            "context_window": {
               "title": "Context Window",
               "description": "The maximum number of context tokens for the model.",
               "type": "integer"
            },
            "model_description": {
               "title": "Model Description",
               "description": "The model description from Xinference.",
               "type": "object"
            }
         },
         "required": [
            "model_uid",
            "endpoint",
            "temperature",
            "max_tokens",
            "context_window",
            "model_description"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "context_window (int)"
      * "endpoint (str)"
      * "max_tokens (int)"
      * "model_description (Dict[str, Any])"
      * "model_uid (str)"
      * "temperature (float)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field context_window: int [Required]
      The maximum number of context tokens for the model.
   field endpoint: str [Required]
      The Xinference endpoint URL to use.
   field max_tokens: int [Required]
      The maximum new tokens to generate as answer.
   field model_description: Dict[str, Any] [Required]
      The model description from Xinference.
   field model_uid: str [Required]
      The Xinference model to use.
   field temperature: float [Required]
      The temperature to use for sampling.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   load_model(model_uid: str, endpoint: str) -> Tuple[Any, int, dict]
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


LlamaCPP

pydantic model llama_index.llms.llama_cpp.LlamaCPP
      {
         "title": "LlamaCPP",
         "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `complete`,\n    `stream_complete`, and `metadata` methods.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model_url": {
               "title": "Model Url",
               "description": "The URL llama-cpp model to download and use.",
               "type": "string"
            },
            "model_path": {
               "title": "Model Path",
               "description": "The path to the llama-cpp model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use for sampling.",
               "type": "number"
            },
            "max_new_tokens": {
               "title": "Max New Tokens",
               "description": "The maximum number of tokens to generate.",
               "type": "integer"
            },
            "context_window": {
               "title": "Context Window",
               "description": "The maximum number of context tokens for the model.",
               "default": 3900,
               "type": "integer"
            },
            "generate_kwargs": {
               "title": "Generate Kwargs",
               "description": "Kwargs used for generation.",
               "type": "object"
            },
            "model_kwargs": {
               "title": "Model Kwargs",
               "description": "Kwargs used for model initialization.",
               "type": "object"
            },
            "verbose": {
               "title": "Verbose",
               "description": "Whether to print verbose output.",
               "default": true,
               "type": "boolean"
            }
         },
         "required": [
            "temperature",
            "max_new_tokens"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "completion_to_prompt (Callable)"
      * "context_window (int)"
      * "generate_kwargs (Dict[str, Any])"
      * "max_new_tokens (int)"
      * "messages_to_prompt (Callable)"
      * "model_kwargs (Dict[str, Any])"
      * "model_path (Optional[str])"
      * "model_url (Optional[str])"
      * "temperature (float)"
      * "verbose (bool)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field completion_to_prompt: Callable [Required]
      The function to convert a completion to a prompt.
   field context_window: int = 3900
      The maximum number of context tokens for the model.
   field generate_kwargs: Dict[str, Any] [Optional]
      Kwargs used for generation.
   field max_new_tokens: int [Required]
      The maximum number of tokens to generate.
   field messages_to_prompt: Callable [Required]
      The function to convert messages to a prompt.
   field model_kwargs: Dict[str, Any] [Optional]
      Kwargs used for model initialization.
   field model_path: Optional[str] = None
      The path to the llama-cpp model to use.
   field model_url: Optional[str] = None
      The URL llama-cpp model to download and use.
   field temperature: float [Required]
      The temperature to use for sampling.
   field verbose: bool = True
      Whether to print verbose output.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against

LlamaCPP

      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Gradient Model Adapter

pydantic model llama_index.llms.gradient.GradientModelAdapterLLM
      {
         "title": "GradientModelAdapterLLM",
         "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `complete`,\n    `stream_complete`, and `metadata` methods.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The number of tokens to generate.",
               "exclusiveMinimum": 0,
               "exclusiveMaximum": 512,
               "type": "integer"
            },
            "access_token": {
               "title": "Access Token",
               "description": "The Gradient access token to use.",
               "type": "string"
            },
            "host": {
               "title": "Host",
               "description": "The url of the Gradient service to access.",
               "type": "string"
            },
            "workspace_id": {
               "title": "Workspace Id",
               "description": "The Gradient workspace id to use.",
               "type": "string"
            },
            "model_adapter_id": {
               "title": "Model Adapter Id",
               "description": "The id of the model adapter to use.",
               "type": "string"
            }
         },
         "required": [
            "model_adapter_id"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "access_token (Optional[str])"
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "host (Optional[str])"
      * "max_tokens (Optional[int])"
      * "model_adapter_id (str)"
      * "workspace_id (Optional[str])"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field access_token: Optional[str] = None
      The Gradient access token to use.
   field host: Optional[str] = None
      The url of the Gradient service to access.
   field max_tokens: Optional[int] = None
      The number of tokens to generate.
      Constraints:
         * **exclusiveMinimum** = 0
         * **exclusiveMaximum** = 512
   field model_adapter_id: str [Required]
      The id of the model adapter to use.
   field workspace_id: Optional[str] = None
      The Gradient workspace id to use.
   close() -> None
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_complete(prompt: str, **kwargs: Any) -> Generator[CompletionResponse, None, None]
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Replicate

pydantic model llama_index.llms.replicate.Replicate
      {
         "title": "Replicate",
         "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `complete`,\n    `stream_complete`, and `metadata` methods.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "model": {
               "title": "Model",
               "description": "The Replicate model to use.",
               "type": "string"
            },
            "temperature": {
               "title": "Temperature",
               "description": "The temperature to use for sampling.",
               "type": "number"
            },
            "context_window": {
               "title": "Context Window",
               "description": "The maximum number of context tokens for the model.",
               "type": "integer"
            },
            "prompt_key": {
               "title": "Prompt Key",
               "description": "The key to use for the prompt in API calls.",
               "type": "string"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "description": "Additional kwargs for the Replicate API.",
               "type": "object"
            }
         },
         "required": [
            "model",
            "temperature",
            "context_window",
            "prompt_key"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "additional_kwargs (Dict[str, Any])"
      * "context_window (int)"
      * "model (str)"
      * "prompt_key (str)"
      * "temperature (float)"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field additional_kwargs: Dict[str, Any] [Optional]
      Additional kwargs for the Replicate API.
   field context_window: int [Required]
      The maximum number of context tokens for the model.
   field model: str [Required]
      The Replicate model to use.
   field prompt_key: str [Required]
      The key to use for the prompt in API calls.
   field temperature: float [Required]
      The temperature to use for sampling.
   chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Chat endpoint for LLM.
   classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Any
      Streaming chat endpoint for LLM.
   stream_complete(*args: Any, **kwargs: Any) -> Any
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Gradient Base Model

pydantic model llama_index.llms.gradient.GradientBaseModelLLM
      {
         "title": "GradientBaseModelLLM",
         "description": "Simple abstract base class for custom LLMs.\n\nSubclasses must implement the `__init__`, `complete`,\n    `stream_complete`, and `metadata` methods.",
         "type": "object",
         "properties": {
            "callback_manager": {
               "title": "Callback Manager"
            },
            "max_tokens": {
               "title": "Max Tokens",
               "description": "The number of tokens to generate.",
               "exclusiveMinimum": 0,
               "exclusiveMaximum": 512,
               "type": "integer"
            },
            "access_token": {
               "title": "Access Token",
               "description": "The Gradient access token to use.",
               "type": "string"
            },
            "host": {
               "title": "Host",
               "description": "The url of the Gradient service to access.",
               "type": "string"
            },
            "workspace_id": {
               "title": "Workspace Id",
               "description": "The Gradient workspace id to use.",
               "type": "string"
            },
            "base_model_slug": {
               "title": "Base Model Slug",
               "description": "The slug of the base model to use.",
               "type": "string"
            }
         },
         "required": [
            "base_model_slug"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "access_token (Optional[str])"
      * "base_model_slug (str)"
      * "callback_manager
        (llama_index.callbacks.base.CallbackManager)"
      * "host (Optional[str])"
      * "max_tokens (Optional[int])"
      * "workspace_id (Optional[str])"
   Validators:
      * "_validate_callback_manager" » "callback_manager"
   field access_token: Optional[str] = None
      The Gradient access token to use.
   field base_model_slug: str [Required]
      The slug of the base model to use.
   field host: Optional[str] = None
      The url of the Gradient service to access.
   field max_tokens: Optional[int] = None
      The number of tokens to generate.
      Constraints:
         * **exclusiveMinimum** = 0
         * **exclusiveMaximum** = 512
   field workspace_id: Optional[str] = None
      The Gradient workspace id to use.
   close() -> None
   complete(*args: Any, **kwargs: Any) -> Any
      Completion endpoint for LLM.
   stream_complete(prompt: str, **kwargs: Any) -> Generator[CompletionResponse, None, None]
      Streaming completion endpoint for LLM.
   property metadata: LLMMetadata
      LLM metadata.


Query Engines

Below we show some general query engine classes.
General Query Engines
^^^^^^^^^^^^^^^^^^^^^
* Graph Query Engine
* Multistep Query Engine
* Retriever Query Engine
* Transform Query Engine
* Router Query Engine
* Retriever Router Query Engine
* Sub Question Query Engine
* SQL Join Query Engine
* Flare Query Engine
* Citation Query Engine
* Knowledge Graph Query Engine
We also show query engine classes specific to our structured indices.
Structured Indices Query Engines
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* SQL Query Engine
* Pandas Query Engine


Response Synthesizer

Init file.
class llama_index.response_synthesizers.Accumulate(text_qa_template: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, output_cls: Optional[Any] = None, streaming: bool = False, use_async: bool = False)
   Accumulate responses from multiple text chunks.
   async aget_response(query_str: str, text_chunks: Sequence[str], separator: str = '\n---------------------\n', **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Apply the same prompt to text chunks and return async responses.
   get_response(query_str: str, text_chunks: Sequence[str], separator: str = '\n---------------------\n', **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Apply the same prompt to text chunks and return responses.
class llama_index.response_synthesizers.BaseSynthesizer(service_context: Optional[ServiceContext] = None, streaming: bool = False, output_cls: BaseModel = None)
   Response builder class.
   abstract async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
   abstract get_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
class llama_index.response_synthesizers.CompactAndRefine(service_context: Optional[ServiceContext] = None, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, output_cls: Optional[BaseModel] = None, streaming: bool = False, verbose: bool = False, structured_answer_filtering: bool = False, program_factory: Optional[Callable[[BasePromptTemplate], BasePydanticProgram]] = None)
   Refine responses across compact text chunks.
   async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
   get_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get compact response.
class llama_index.response_synthesizers.Generation(simple_template: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, streaming: bool = False)
   async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
   get_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
class llama_index.response_synthesizers.Refine(service_context: Optional[ServiceContext] = None, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, output_cls: Optional[BaseModel] = None, streaming: bool = False, verbose: bool = False, structured_answer_filtering: bool = False, program_factory: Optional[Callable[[BasePromptTemplate], BasePydanticProgram]] = None)
   Refine a response to a query across text chunks.
   async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
   get_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]

Response Synthesizer

      Give response over chunks.
class llama_index.response_synthesizers.ResponseMode(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
   Response modes of the response builder (and synthesizer).
   ACCUMULATE = 'accumulate'
      Synthesize a response for each text chunk, and then return the
      concatenation.
   COMPACT = 'compact'
      Compact and refine mode first combine text chunks into larger
      consolidated chunks     that more fully utilize the available
      context window, then refine answers     across them. This mode
      is faster than refine since we make fewer calls to the LLM.
   COMPACT_ACCUMULATE = 'compact_accumulate'
      Compact and accumulate mode first combine text chunks into
      larger consolidated     chunks that more fully utilize the
      available context window, then accumulate     answers for each
      of them and finally return the concatenation. This mode is
      faster than accumulate since we make fewer calls to the LLM.
   GENERATION = 'generation'
      Ignore context, just use LLM to generate a response.
   NO_TEXT = 'no_text'
      Return the retrieved context nodes, without synthesizing a final
      response.
   REFINE = 'refine'
      Refine is an iterative way of generating a response. We first
      use the context in the first node, along with the query, to
      generate an     initial answer. We then pass this answer, the
      query, and the context of the second node as input     into a
      “refine prompt” to generate a refined answer. We refine through
      N-1 nodes,     where N is the total number of nodes.
   SIMPLE_SUMMARIZE = 'simple_summarize'
      Merge all text chunks into one, and make a LLM call. This will
      fail if the merged text chunk exceeds the context window size.
   TREE_SUMMARIZE = 'tree_summarize'
      Build a tree index over the set of candidate nodes, with a
      summary prompt seeded     with the query. The tree is built in a
      bottoms-up fashion, and in the end the root node is     returned
      as the response
   capitalize()
      Return a capitalized version of the string.
      More specifically, make the first character have upper case and
      the rest lower case.
   casefold()
      Return a version of the string suitable for caseless
      comparisons.
   center(width, fillchar=' ', /)
      Return a centered string of length width.
      Padding is done using the specified fill character (default is a
      space).
   count(sub[, start[, end]]) -> int
      Return the number of non-overlapping occurrences of substring
      sub in string S[start:end].  Optional arguments start and end
      are interpreted as in slice notation.
   encode(encoding='utf-8', errors='strict')
      Encode the string using the codec registered for encoding.
      encoding
         The encoding in which to encode the string.
      errors
         The error handling scheme to use for encoding errors. The
         default is 'strict' meaning that encoding errors raise a
         UnicodeEncodeError.  Other possible values are 'ignore',
         'replace' and 'xmlcharrefreplace' as well as any other name
         registered with codecs.register_error that can handle
         UnicodeEncodeErrors.
   endswith(suffix[, start[, end]]) -> bool
      Return True if S ends with the specified suffix, False
      otherwise. With optional start, test S beginning at that
      position. With optional end, stop comparing S at that position.
      suffix can also be a tuple of strings to try.
   expandtabs(tabsize=8)
      Return a copy where all tab characters are expanded using
      spaces.
      If tabsize is not given, a tab size of 8 characters is assumed.

Response Synthesizer

   find(sub[, start[, end]]) -> int
      Return the lowest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Return -1 on failure.
   format(*args, **kwargs) -> str
      Return a formatted version of S, using substitutions from args
      and kwargs. The substitutions are identified by braces ('{' and
      '}').
   format_map(mapping) -> str
      Return a formatted version of S, using substitutions from
      mapping. The substitutions are identified by braces ('{' and
      '}').
   index(sub[, start[, end]]) -> int
      Return the lowest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Raises ValueError when the substring is not found.
   isalnum()
      Return True if the string is an alpha-numeric string, False
      otherwise.
      A string is alpha-numeric if all characters in the string are
      alpha-numeric and there is at least one character in the string.
   isalpha()
      Return True if the string is an alphabetic string, False
      otherwise.
      A string is alphabetic if all characters in the string are
      alphabetic and there is at least one character in the string.
   isascii()
      Return True if all characters in the string are ASCII, False
      otherwise.
      ASCII characters have code points in the range U+0000-U+007F.
      Empty string is ASCII too.
   isdecimal()
      Return True if the string is a decimal string, False otherwise.
      A string is a decimal string if all characters in the string are
      decimal and there is at least one character in the string.
   isdigit()
      Return True if the string is a digit string, False otherwise.
      A string is a digit string if all characters in the string are
      digits and there is at least one character in the string.
   isidentifier()
      Return True if the string is a valid Python identifier, False
      otherwise.
      Call keyword.iskeyword(s) to test whether string s is a reserved
      identifier, such as "def" or "class".
   islower()
      Return True if the string is a lowercase string, False
      otherwise.
      A string is lowercase if all cased characters in the string are
      lowercase and there is at least one cased character in the
      string.
   isnumeric()
      Return True if the string is a numeric string, False otherwise.
      A string is numeric if all characters in the string are numeric
      and there is at least one character in the string.
   isprintable()
      Return True if the string is printable, False otherwise.
      A string is printable if all of its characters are considered
      printable in repr() or if it is empty.
   isspace()
      Return True if the string is a whitespace string, False
      otherwise.
      A string is whitespace if all characters in the string are
      whitespace and there is at least one character in the string.
   istitle()
      Return True if the string is a title-cased string, False
      otherwise.
      In a title-cased string, upper- and title-case characters may
      only follow uncased characters and lowercase characters only
      cased ones.
   isupper()
      Return True if the string is an uppercase string, False
      otherwise.
      A string is uppercase if all cased characters in the string are
      uppercase and there is at least one cased character in the
      string.
   join(iterable, /)
      Concatenate any number of strings.
      The string whose method is called is inserted in between each

Response Synthesizer

      given string. The result is returned as a new string.
      Example: '.'.join(['ab', 'pq', 'rs']) -> 'ab.pq.rs'
   ljust(width, fillchar=' ', /)
      Return a left-justified string of length width.
      Padding is done using the specified fill character (default is a
      space).
   lower()
      Return a copy of the string converted to lowercase.
   lstrip(chars=None, /)
      Return a copy of the string with leading whitespace removed.
      If chars is given and not None, remove characters in chars
      instead.
   static maketrans()
      Return a translation table usable for str.translate().
      If there is only one argument, it must be a dictionary mapping
      Unicode ordinals (integers) or characters to Unicode ordinals,
      strings or None. Character keys will be then converted to
      ordinals. If there are two arguments, they must be strings of
      equal length, and in the resulting dictionary, each character in
      x will be mapped to the character at the same position in y. If
      there is a third argument, it must be a string, whose characters
      will be mapped to None in the result.
   partition(sep, /)
      Partition the string into three parts using the given separator.
      This will search for the separator in the string.  If the
      separator is found, returns a 3-tuple containing the part before
      the separator, the separator itself, and the part after it.
      If the separator is not found, returns a 3-tuple containing the
      original string and two empty strings.
   removeprefix(prefix, /)
      Return a str with the given prefix string removed if present.
      If the string starts with the prefix string, return
      string[len(prefix):]. Otherwise, return a copy of the original
      string.
   removesuffix(suffix, /)
      Return a str with the given suffix string removed if present.
      If the string ends with the suffix string and that suffix is not
      empty, return string[:-len(suffix)]. Otherwise, return a copy of
      the original string.
   replace(old, new, count=-1, /)
      Return a copy with all occurrences of substring old replaced by
      new.
         count
            Maximum number of occurrences to replace. -1 (the default
            value) means replace all occurrences.
      If the optional argument count is given, only the first count
      occurrences are replaced.
   rfind(sub[, start[, end]]) -> int
      Return the highest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Return -1 on failure.
   rindex(sub[, start[, end]]) -> int
      Return the highest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Raises ValueError when the substring is not found.
   rjust(width, fillchar=' ', /)
      Return a right-justified string of length width.
      Padding is done using the specified fill character (default is a
      space).
   rpartition(sep, /)
      Partition the string into three parts using the given separator.
      This will search for the separator in the string, starting at
      the end. If the separator is found, returns a 3-tuple containing
      the part before the separator, the separator itself, and the
      part after it.
      If the separator is not found, returns a 3-tuple containing two
      empty strings and the original string.
   rsplit(sep=None, maxsplit=-1)
      Return a list of the substrings in the string, using sep as the

Response Synthesizer

      separator string.
         sep
            The separator used to split the string.
            When set to None (the default value), will split on any
            whitespace character (including n r t f and spaces) and
            will discard empty strings from the result.
         maxsplit
            Maximum number of splits (starting from the left). -1 (the
            default value) means no limit.
      Splitting starts at the end of the string and works to the
      front.
   rstrip(chars=None, /)
      Return a copy of the string with trailing whitespace removed.
      If chars is given and not None, remove characters in chars
      instead.
   split(sep=None, maxsplit=-1)
      Return a list of the substrings in the string, using sep as the
      separator string.
         sep
            The separator used to split the string.
            When set to None (the default value), will split on any
            whitespace character (including n r t f and spaces) and
            will discard empty strings from the result.
         maxsplit
            Maximum number of splits (starting from the left). -1 (the
            default value) means no limit.
      Note, str.split() is mainly useful for data that has been
      intentionally delimited.  With natural text that includes
      punctuation, consider using the regular expression module.
   splitlines(keepends=False)
      Return a list of the lines in the string, breaking at line
      boundaries.
      Line breaks are not included in the resulting list unless
      keepends is given and true.
   startswith(prefix[, start[, end]]) -> bool
      Return True if S starts with the specified prefix, False
      otherwise. With optional start, test S beginning at that
      position. With optional end, stop comparing S at that position.
      prefix can also be a tuple of strings to try.
   strip(chars=None, /)
      Return a copy of the string with leading and trailing whitespace
      removed.
      If chars is given and not None, remove characters in chars
      instead.
   swapcase()
      Convert uppercase characters to lowercase and lowercase
      characters to uppercase.
   title()
      Return a version of the string where each word is titlecased.
      More specifically, words start with uppercased characters and
      all remaining cased characters have lower case.
   translate(table, /)
      Replace each character in the string using the given translation
      table.
         table
            Translation table, which must be a mapping of Unicode
            ordinals to Unicode ordinals, strings, or None.
      The table must implement lookup/indexing via __getitem__, for
      instance a dictionary or list.  If this operation raises
      LookupError, the character is left untouched.  Characters mapped
      to None are deleted.
   upper()
      Return a copy of the string converted to uppercase.
   zfill(width, /)
      Pad a numeric string with zeros on the left, to fill a field of
      the given width.
      The string is never truncated.
class llama_index.response_synthesizers.SimpleSummarize(text_qa_template: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, streaming: bool = False)
   async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
   get_response(query_str: str, text_chunks: Sequence[str], **kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get response.
class llama_index.response_synthesizers.TreeSummarize(summary_template: Optional[BasePromptTemplate] = None, service_context: Optional[ServiceContext] = None, output_cls: Optional[BaseModel] = None, streaming: bool = False, use_async: bool = False, verbose: bool = False)

Response Synthesizer

   Tree summarize response builder.
   This response builder recursively merges text chunks and summarizes
   them in a bottom-up fashion (i.e. building a tree from leaves to
   root).
   More concretely, at each recursively step: 1. we repack the text
   chunks so that each chunk fills the context window of the LLM 2. if
   there is only one chunk, we give the final response 3. otherwise,
   we summarize each chunk and recursively summarize the summaries.
   async aget_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get tree summarize response.
   get_response(query_str: str, text_chunks: Sequence[str], **response_kwargs: Any) -> Union[BaseModel, str, Generator[str, None, None]]
      Get tree summarize response.
llama_index.response_synthesizers.get_response_synthesizer(service_context: Optional[ServiceContext] = None, text_qa_template: Optional[BasePromptTemplate] = None, refine_template: Optional[BasePromptTemplate] = None, summary_template: Optional[BasePromptTemplate] = None, simple_template: Optional[BasePromptTemplate] = None, response_mode: ResponseMode = ResponseMode.COMPACT, callback_manager: Optional[CallbackManager] = None, use_async: bool = False, streaming: bool = False, structured_answer_filtering: bool = False, output_cls: Optional[BaseModel] = None, program_factory: Optional[Callable[[PromptTemplate], BasePydanticProgram]] = None, verbose: bool = False) -> BaseSynthesizer
   Get a response synthesizer.


Query Bundle

Query Schema.
This schema is used under the hood for all queries, but is primarily
exposed for recursive queries over composable indices.
class llama_index.indices.query.schema.QueryBundle(query_str: str, custom_embedding_strs: Optional[List[str]] = None, embedding: Optional[List[float]] = None)
   Query bundle.
   This dataclass contains the original query string and associated
   transformations.
   Parameters:
      * **query_str** (*str*) -- the original user-specified query
        string. This is currently used by all non embedding-based
        queries.
      * **embedding_strs** (*list**[**str**]*) -- list of strings used
        for embedding the query. This is currently used by all
        embedding-based queries.
      * **embedding** (*list**[**float**]*) -- the stored embedding
        for the query.
   property embedding_strs: List[str]
      Use custom embedding strs if specified, otherwise use query str.


Query Transform

Query Transforms.
class llama_index.indices.query.query_transform.DecomposeQueryTransform(llm_predictor: Optional[BaseLLMPredictor] = None, decompose_query_prompt: Optional[PromptTemplate] = None, verbose: bool = False)
   Decompose query transform.
   Decomposes query into a subquery given the current index struct.
   Performs a single step transformation.
   Parameters:
      **llm_predictor** (*Optional**[**LLMPredictor**]*) -- LLM for
      generating hypothetical documents
   run(query_bundle_or_str: Union[str, QueryBundle], metadata: Optional[Dict] = None) -> QueryBundle
      Run query transform.
class llama_index.indices.query.query_transform.HyDEQueryTransform(llm_predictor: Optional[BaseLLMPredictor] = None, hyde_prompt: Optional[BasePromptTemplate] = None, include_original: bool = True)
   Hypothetical Document Embeddings (HyDE) query transform.
   It uses an LLM to generate hypothetical answer(s) to a given query,
   and use the resulting documents as embedding strings.
   As described in *[Precise Zero-Shot Dense Retrieval without
   Relevance Labels] (https://arxiv.org/abs/2212.10496)*
   run(query_bundle_or_str: Union[str, QueryBundle], metadata: Optional[Dict] = None) -> QueryBundle
      Run query transform.
class llama_index.indices.query.query_transform.StepDecomposeQueryTransform(llm_predictor: Optional[BaseLLMPredictor] = None, step_decompose_query_prompt: Optional[PromptTemplate] = None, verbose: bool = False)
   Step decompose query transform.
   Decomposes query into a subquery given the current index struct and
   previous reasoning.
   NOTE: doesn't work yet.
   Parameters:
      **llm_predictor** (*Optional**[**LLMPredictor**]*) -- LLM for
      generating hypothetical documents
   run(query_bundle_or_str: Union[str, QueryBundle], metadata: Optional[Dict] = None) -> QueryBundle
      Run query transform.