Documentation Guide

A guide for docs contributors
The "docs" directory contains the sphinx source text for LlamaIndex
docs, visit https://gpt-index.readthedocs.io/ to read the full
documentation.
This guide is made for anyone who's interested in running LlamaIndex
documentation locally, making changes to it and make contributions.
LlamaIndex is made by the thriving community behind it, and you're
always welcome to make contributions to the project and the
documentation.
Build Docs
If you haven't already, clone the LlamaIndex Github repo to a local
directory:
   git clone https://github.com/jerryjliu/llama_index.git && cd llama_index
Install all dependencies required for building docs (mainly "sphinx"
and its extension):
* Install poetry - this will help you manage package dependencies
* "poetry shell" - this command creates a virtual environment, which
  keeps installed packages contained to this project
* "poetry install --with docs" - this will install all dependencies
  needed for building docs
Build the sphinx docs:
   cd docs
   make html
The docs HTML files are now generated under "docs/_build/html"
directory, you can preview it locally with the following command:
   python -m http.server 8000 -d _build/html
And open your browser at http://0.0.0.0:8000/ to view the generated
docs.
Watch Docs
We recommend using sphinx-autobuild during development, which provides
a live-reloading server, that rebuilds the documentation and refreshes
any open pages automatically when changes are saved. This enables a
much shorter feedback loop which can help boost productivity when
writing documentation.
Simply run the following command from LlamaIndex project's root
directory:
   make watch-docs


Deprecated Terms

As LlamaIndex continues to evolve, many class names and APIs have been
adjusted, improved, and deprecated.
The following is a list of previously popular terms that have been
deprecated, with links to their replacements.
GPTSimpleVectorIndex
This has been renamed to "VectorStoreIndex", as well as unifying all
vector indexes to a single unified interface. You can integrate with
various vector databases by modifying the underlying "vector_store".
Please see the following links for more details on usage.
* Index Usage Pattern
* Vector Store Guide
* Vector Store Integrations
GPTVectorStoreIndex
This has been renamed to "VectorStoreIndex", but it is only a cosmetic
change. Please see the following links for more details on usage.
* Index Usage Pattern
* Vector Store Guide
* Vector Store Integrations
LLMPredictor
The "LLMPredictor" object is no longer intended to be used by users.
Instead, you can setup an LLM directly and pass it into the
"ServiceContext".
* LLMs in LlamaIndex
* Setting LLMs in the ServiceContext
PromptHelper and max_input_size/
The "max_input_size" parameter for the prompt helper has since been
replaced with "context_window".
The "PromptHelper" in general has been deprecated in favour of
specifying parameters directly in the "service_context" and
"node_parser".
See the following links for more details.
* Configuring settings in the Service Context
* Parsing Documents into Nodes


Welcome to LlamaIndex 🦙 !

LlamaIndex (formerly GPT Index) is a data framework for LLM
applications to ingest, structure, and access private or domain-
specific data.
🚀 Why LlamaIndex?
At their core, LLMs offer a natural language interface between humans
and inferred data. Widely available models come pre-trained on huge
amounts of publicly available data, from Wikipedia and mailing lists
to textbooks and source code.
Applications built on top of LLMs often require augmenting these
models with private or domain-specific data. Unfortunately, that data
can be distributed across siloed applications and data stores. It's
behind APIs, in SQL databases, or trapped in PDFs and slide decks.
That's where **LlamaIndex** comes in.
🦙 How can LlamaIndex help?
LlamaIndex provides the following tools:
* **Data connectors** ingest your existing data from their native
  source and format. These could be APIs, PDFs, SQL, and (much) more.
* **Data indexes** structure your data in intermediate representations
  that are easy and performant for LLMs to consume.
* **Engines** provide natural language access to your data. For
  example:
  * Query engines are powerful retrieval interfaces for knowledge-
    augmented output.
  * Chat engines are conversational interfaces for multi-message,
    "back and forth" interactions with your data.
* **Data agents** are LLM-powered knowledge workers augmented by
  tools, from simple helper functions to API integrations and more.
* **Application integrations** tie LlamaIndex back into the rest of
  your ecosystem. This could be LangChain, Flask, Docker, ChatGPT, or…
  anything else!
👨👩👧👦 Who is LlamaIndex for?
LlamaIndex provides tools for beginners, advanced users, and everyone
in between.
Our high-level API allows beginner users to use LlamaIndex to ingest
and query their data in 5 lines of code.
For more complex applications, our lower-level APIs allow advanced
users to customize and extend any module—data connectors, indices,
retrievers, query engines, reranking modules—to fit their needs.
Getting Started
"pip install llama-index"
Our documentation includes detailed Installation Instructions and a
Starter Tutorial to build your first application (in five lines of
code!)
Once you're up and running, High-Level Concepts has an overview of
LlamaIndex's modular architecture. For more hands-on practical
examples, look through our End-to-End Tutorials or learn how to
customize components to fit your specific needs.
**NOTE**: We have a Typescript package too! Repo, Docs
🗺 Ecosystem
To download or contribute, find LlamaIndex on:
* Github: https://github.com/jerryjliu/llama_index
* PyPi:
  * LlamaIndex: https://pypi.org/project/llama-index/.
  * GPT Index (duplicate): https://pypi.org/project/gpt-index/.
* NPM (Typescript/Javascript):
     * Github: https://github.com/run-llama/LlamaIndexTS
     * Docs: https://ts.llamaindex.ai/
     * LlamaIndex.TS: https://www.npmjs.com/package/llamaindex
Community
Need help? Have a feature suggestion? Join the LlamaIndex community:
* Twitter: https://twitter.com/llama_index
* Discord https://discord.gg/dGcwcsnxhU
Associated projects
* 🏡 LlamaHub: https://llamahub.ai | A large (and growing!) collection
  of custom data connectors
* 🧪 LlamaLab: https://github.com/run-llama/llama-lab | Ambitious

Welcome to LlamaIndex 🦙 !

projects built on top of LlamaIndex


Contributing to LlamaIndex

Interested in contributing to LlamaIndex? Here's how to get started!
Contribution Guideline
The best part of LlamaIndex is our community of users and
contributors.
What should I work on?
1. 🆕 Extend core modules
2. 🐛 Fix bugs
3. 🎉 Add usage examples
4. 🧪 Add experimental features
5. 📄 Improve code quality & documentation
Also, join our Discord for ideas and discussions:
https://discord.gg/dGcwcsnxhU.
1. 🆕 Extend Core Modules
The most impactful way to contribute to LlamaIndex is extending our
core modules:
[image: LlamaIndex modules][image]
We welcome contributions in *all* modules shown above. So far, we have
implemented a core set of functionalities for each. As a contributor,
you can help each module unlock its full potential.
**NOTE**: We are making rapid improvements to the project, and as a
result, some interfaces are still volatile. Specifically, we are
actively working on making the following components more modular and
extensible (uncolored boxes above): core indexes, document stores,
index queries, query runner
Module Details
~~~~~~~~~~~~~~
Below, we will describe what each module does, give a high-level idea
of the interface, show existing implementations, and give some ideas
for contribution.
Data Loaders
~~~~~~~~~~~~
A data loader ingests data of any format from anywhere into "Document"
objects, which can then be parsed and indexed.
**Interface**: "load_data" takes arbitrary arguments as input (e.g.
path to data), and outputs a sequence of "Document" objects.
**Examples**:
* Google Sheets Loader
* Gmail Loader
* Github Repository Loader
Contributing a data loader is easy and super impactful for the
community. The preferred way to contribute is making a PR at LlamaHub
Github.
**Ideas**
* Want to load something but there's no LlamaHub data loader for it
  yet? Make a PR!
Node Parser
~~~~~~~~~~~
A node parser parses "Document" objects into "Node" objects (atomic
unit of data that LlamaIndex operates over, e.g., chunk of text,
image, or table). It is responsible for splitting text (via text
splitters) and explicitly modelling the relationship between units of
data (e.g. A is the source of B, C is a chunk after D).
**Interface**: "get_nodes_from_documents" takes a sequence of
"Document" objects as input, and outputs a sequence of "Node" objects.
**Examples**:
* Simple Node Parser
See the API reference for full details.
**Ideas**:
* Add new "Node" relationships to model to model hierarchical
  documents (e.g. play-act-scene, chapter-section-heading).
Text Splitters
~~~~~~~~~~~~~~
Text splitter splits a long text "str" into smaller text "str" chunks
with desired size and splitting "strategy" since LLMs have a limited
context window size, and the quality of text chunk used as context
impacts the quality of query results.
**Interface**: "split_text" takes a "str" as input, and outputs a
sequence of "str"
**Examples**:
* Token Text Splitter
* Sentence Splitter
* Code Splitter
Document/Index/KV Stores
~~~~~~~~~~~~~~~~~~~~~~~~
Under the hood, LlamaIndex also supports a swappable **storage layer**
that allows you to customize Document Stores (where ingested documents
(i.e., "Node" objects) are stored), and Index Stores (where index
metadata are stored)
We have an underlying key-value abstraction backing the document/index
stores. Currently we support in-memory and MongoDB storage for these
stores. Open to contributions!
See Storage guide for details.
Managed Index
~~~~~~~~~~~~~

Contributing to LlamaIndex

A managed index is used to represent an index that's managed via an
API, exposing API calls to index documents and query documents.
Currently we support the VectaraIndex. Open to contributions!
See Managed Index docs for details.
Vector Stores
~~~~~~~~~~~~~
Our vector store classes store embeddings and support lookup via
similarity search. These serve as the main data store and retrieval
engine for our vector index.
**Interface**:
* "add" takes in a sequence of "NodeWithEmbeddings" and insert the
  embeddings (and possibly the node contents & metadata) into the
  vector store.
* "delete" removes entries given document IDs.
* "query" retrieves top-k most similar entries given a query
  embedding.
**Examples**:
* Pinecone
* Faiss
* Chroma
**Ideas**:
* See a vector database out there that we don't support yet? Make a
  PR!
See reference for full details.
Retrievers
~~~~~~~~~~
Our retriever classes are lightweight classes that implement a
"retrieve" method. They may take in an index class as input - by
default, each of our indices (list, vector, keyword) have an
associated retriever. The output is a set of "NodeWithScore" objects
(a "Node" object with an extra "score" field).
You may also choose to implement your own retriever classes on top of
your own data if you wish.
**Interface**:
* "retrieve" takes in a "str" or "QueryBundle" as input, and outputs a
  list of "NodeWithScore" objects
**Examples**:
* Vector Index Retriever
* List Index Retriever
* Transform Retriever
**Ideas**:
* Besides the "default" retrievers built on top of each index, what
  about fancier retrievers? E.g. retrievers that take in other
  retrievers as input? Or other types of data?
Query Engines
~~~~~~~~~~~~~
Our query engine classes are lightweight classes that implement a
"query" method; the query returns a response type. For instance, they
may take in a retriever class as input; our "RetrieverQueryEngine"
takes in a "retriever" as input as well as a "BaseSynthesizer" class
for response synthesis, and the "query" method performs retrieval and
synthesis before returning the final result. They may take in other
query engine classes in as input too.
**Interface**:
* "query" takes in a "str" or "QueryBundle" as input, and outputs a
  "Response" object.
**Examples**:
* Retriever Query Engine
* Transform Query Engine
Query Transforms
~~~~~~~~~~~~~~~~
A query transform augments a raw query string with associated
transformations to improve index querying. This can interpreted as a
pre-processing stage, before the core index query logic is executed.
**Interface**: "run" takes in a "str" or "Querybundle" as input, and
outputs a transformed "QueryBundle".
**Examples**:
* Hypothetical Document Embeddings
* Query Decompose
See guide for more information.
Token Usage Optimizers
~~~~~~~~~~~~~~~~~~~~~~
A token usage optimizer refines the retrieved "Nodes" to reduce token
usage during response synthesis.
**Interface**: "optimize" takes in the "QueryBundle" and a text chunk
"str", and outputs a refined text chunk "str" that yields a more
optimized response
**Examples**:
* Sentence Embedding Optimizer
Node Postprocessors
~~~~~~~~~~~~~~~~~~~
A node postprocessor refines a list of retrieve nodes given
configuration and context.
**Interface**: "postprocess_nodes" takes a list of "Nodes" and extra
metadata (e.g. similarity and query), and outputs a refined list of

Contributing to LlamaIndex

"Nodes".
**Examples**:
* Keyword Postprocessor: filters nodes based on keyword match
* Similarity Postprocessor: filers nodes based on similarity threshold
* Prev Next Postprocessor: fetches additional nodes to augment context
  based on node relationships.
Output Parsers
~~~~~~~~~~~~~~
A output parser enables us to extract structured output from the plain
text output generated by the LLM.
**Interface**:
* "format": formats a query "str" with structured output formatting
  instructions, and outputs the formatted "str"
* "parse": takes a "str" (from LLM response) as input, and gives a
  parsed tructured output (optionally also validated, error-
  corrected).
**Examples**:
* Guardrails Output Parser
* Langchain Output Parser
See guide for more information.
2. 🐛 Fix Bugs
Most bugs are reported and tracked in the Github Issues Page. We try
our best in triaging and tagging these issues:
* Issues tagged as "bug" are confirmed bugs.
* New contributors may want to start with issues tagged with "good
  first issue".
Please feel free to open an issue and/or assign an issue to yourself.
3. 🎉 Add Usage Examples
If you have applied LlamaIndex to a unique use-case (e.g. interesting
dataset, customized index structure, complex query), we would love
your contribution in the form of:
1. a guide: e.g. guide to LlamIndex + Structured Data
2. an example notebook: e.g. Composable Indices Demo
4. 🧪 Add Experimental Features
If you have a crazy idea, make a PR for it! Whether if it's the latest
research, or what you thought of in the shower, we'd love to see
creative ways to improve LlamaIndex.
5. 📄 Improve Code Quality & Documentation
We would love your help in making the project cleaner, more robust,
and more understandable. If you find something confusing, it most
likely is for other people as well. Help us be better!
Development Guideline
Environment Setup
LlamaIndex is a Python package. We've tested primarily with Python
versions >= 3.8. Here's a quick and dirty guide to getting your
environment setup.
First, create a fork of LlamaIndex, by clicking the "Fork" button on
the LlamaIndex Github page. Following these steps for more details on
how to fork the repo and clone the forked repo.
Then, create a new Python virtual environment using poetry.
* Install poetry - this will help you manage package dependencies
* "poetry shell" - this command creates a virtual environment, which
  keeps installed packages contained to this project
* "poetry install --with dev,docs" - this will install all
  dependencies needed for most local development
Now you should be set!
Validating your Change
Let's make sure to "format/lint" our change. For bigger changes, let's
also make sure to "test" it and perhaps create an "example notebook".
Formatting/Linting
~~~~~~~~~~~~~~~~~~
You can format and lint your changes with the following commands in
the root directory:
   make format; make lint
You can also make use of our pre-commit hooks by setting up git hook
scripts:
   pre-commit install
We run an assortment of linters: "black", "ruff", "mypy".
Testing
~~~~~~~
For bigger changes, you'll want to create a unit test. Our tests are
in the "tests" folder. We use "pytest" for unit testing. To run all
unit tests, run the following in the root dir:
   pip install -r data_requirements.txt
   pytest tests
Creating an Example Notebook
For changes that involve entirely new features, it may be worth adding

Contributing to LlamaIndex

an example Jupyter notebook to showcase this feature.
Example notebooks can be found in this folder:
https://github.com/jerryjliu/llama_index/tree/main/examples.
Creating a pull request
See these instructions to open a pull request against the main
LlamaIndex repo.


Documentation Guide


ChangeLog

[0.8.45] - 2023-10-13
New Features
* Added support for fine-tuning cross encoders (#7705)
* Added "QueryFusionRetriever" for merging multiple retrievers + query
  augmentation (#8100)
* Added "nb-clean" to "pre-commit" to minimize PR diffs (#8108)
* Support for "TextEmbeddingInference" embeddings (#8122)
Bug Fixes / Nits
* Improved the "BM25Retriever" interface to accept "BaseNode" objects
  (#8096)
* Fixed bug with "BM25Retriever" tokenizer not working as expected
  (#8096)
* Brought mypy to pass in Python 3.8 (#8107)
[0.8.44] - 2023-10-12
New Features
* add pgvector sql query engine (#8087)
* Added HoneyHive one-click observability (#7944)
* Add support for both SQLAlchemy V1 and V2 (#8060)
[0.8.43.post1] - 2023-10-11
New Features
* Moves "codespell" to "pre-commit" (#8040)
* Added "prettier" for autoformatting extensions besides ".py" (#8072)
Bug Fixes / Nits
* Fixed forgotten f-str in "HuggingFaceLLM" (#8075)
* Relaxed numpy/panadas reqs
[0.8.43] - 2023-10-10
New Features
* Added support for "GradientEmbedding" embed models (#8050)
Bug Fixes / Nits
* added "messages_to_prompt" kwarg to "HuggingFaceLLM" (#8054)
* improved selection and sql parsing for open-source models (#8054)
* fixed bug when agents hallucinate too many kwargs for a tool (#8054)
* improved prompts and debugging for selection+question generation
  (#8056)
[0.8.42] - 2023-10-10
New Features
* "LocalAI" more intuitive module-level var names (#8028)
* Enable "codespell" for markdown docs (#7972)
* add unstructured table element node parser (#8036)
* Add: Async upserting for Qdrant vector store (#7968)
* Add cohere llm (#8023)
Bug Fixes / Nits
* Parse multi-line outputs in react agent answers (#8029)
* Add properly named kwargs to keyword "as_retriever" calls (#8011)
* Updating Reference to RAGAS LlamaIndex Integration (#8035)
* Vectara bugfix (#8032)
* Fix: ChromaVectorStore can attempt to add in excess of chromadb
  batch… (#8019)
* Fix get_content method in Mbox reader (#8012)
* Apply kwarg filters in WeaviateVectorStore (#8017)
* Avoid ZeroDivisionError (#8027)
* "LocalAI" intuitive module-level var names (#8028)
* zep/fix: imports & typing (#8030)
* refactor: use "str.join" (#8020)
* use proper metadata str for node parsing (#7987)
[0.8.41] - 2023-10-07
New Features
* You.com retriever (#8024)
* Pull fields from mongodb into metadata with "metadata_names"
  argument (#8001)
* Simplified "LocalAI.__init__" preserving the same behaviors (#7982)
Bug Fixes / Nits
* Use longest metadata string for metadata aware text splitting
  (#7987)
* Handle lists of strings in mongodb reader (#8002)
* Removes "OpenAI.class_type" as it was dead code (#7983)
* Fixing "HuggingFaceLLM.device_map" type hint (#7989)

ChangeLog

[0.8.40] - 2023-10-05
New Features
* Added support for "Clarifai" LLM (#7967)
* Add support for function fine-tuning (#7971)
Breaking Changes
* Update document summary index (#7815)
  * change default retrieval mode to embedding
  * embed summaries into vector store by default at indexing time
    (instead of calculating embedding on the fly)
  * support configuring top k in llm retriever
[0.8.39] - 2023-10-03
New Features
* Added support for pydantic object outputs with query engines (#7893)
* "ClarifaiEmbedding" class added for embedding support (#7940)
* Markdown node parser, flat file reader and simple file node parser
  (#7863)
* Added support for mongdb atlas "$vectorSearch" (#7866)
Bug Fixes / Nits
* Adds support for using message metadata in discord reader (#7906)
* Fix "LocalAI" chat capability without "max_tokens" (#7942)
* Added "codespell" for automated checking (#7941)
* "ruff" modernization and autofixes (#7889)
* Implement own SQLDatabase class (#7929)
* Update LlamaCPP context_params property (#7945)
* fix duplicate embedding (#7949)
* Adds "codespell" tool for enforcing good spelling (#7941)
* Supporting "mypy" local usage with "venv" (#7952)
* Vectara - minor update (#7954)
* Avoiding "pydantic" reinstalls in CI (#7956)
* move tree_sitter_languages into data_requirements.txt (#7955)
* Add "cache_okay" param to "PGVectorStore" to help suppress TSVector
  warnings (#7950)
[0.8.38] - 2023-10-02
New Features
* Updated "KeywordNodePostprocessor" to use spacy to support more
  languages (#7894)
* "LocalAI" supporting global or per-query "/chat/completions" vs
  "/completions" (#7921)
* Added notebook on using REBEL + Wikipedia filtering for knowledge
  graphs (#7919)
* Added support for "ElasticsearchEmbeddings" (#7914)
[0.8.37] - 2023-09-30
New Features
* Supporting "LocalAI" LLMs (#7913)
* Validations protecting against misconfigured chunk sizes (#7917)
Bug Fixes / Nits
* Simplify NL SQL response to SQL parsing, with expanded NL SQL prompt
  (#7868)
* Improve vector store retrieval speed for vectordb integrations
  (#7876)
* Added replacing {{ and }}, and fixed JSON parsing recursion (#7888)
* Nice-ified JSON decoding error (#7891)
* Nice-ified SQL error from LLM not providing SQL (#7900)
* Nice-ified "ImportError" for "HuggingFaceLLM" (#7904)
* eval fixes: fix dataset response generation, add score to evaluators
  (#7915)
[0.8.36] - 2023-09-27
New Features
* add "build RAG from scratch notebook" - OSS/local (#7864)
Bug Fixes / Nits
* Fix elasticsearch hybrid scoring (#7852)
* Replace "get_color_mapping" and "print_text" Langchain dependency
  with internal implementation (#7845)
* Fix async streaming with azure (#7856)
* Avoid "NotImplementedError()" in sub question generator (#7855)
* Patch predibase initialization (#7859)
* Bumped min langchain version and changed prompt imports from
  langchain (#7862)
[0.8.35] - 2023-09-27

ChangeLog

Bug Fixes / Nits
* Fix dropping textnodes in recursive retriever (#7840)
* share callback_manager between agent and its llm when
  callback_manager is None (#7844)
* fix pandas query engine (#7847)
[0.8.34] - 2023-09-26
New Features
* Added "Konko" LLM support (#7775)
* Add before/after context sentence (#7821)
* EverlyAI integration with LlamaIndex through OpenAI library (#7820)
* add Arize Phoenix tracer to global handlers (#7835)
Bug Fixes / Nits
* Normalize scores returned from ElasticSearch vector store (#7792)
* Fixed "refresh_ref_docs()" bug with order of operations (#7664)
* Delay postgresql connection for "PGVectorStore" until actually
  needed (#7793)
* Fix KeyError in delete method of "SimpleVectorStore" related to
  metadata filters (#7829)
* Fix KeyError in delete method of "SimpleVectorStore" related to
  metadata filters (#7831)
* Addressing PyYAML import error (#7784)
* ElasticsearchStore: Update User-Agent + Add example docker compose
  (#7832)
* "StorageContext.persist" supporting "Path" (#7783)
* Update ollama.py (#7839)
* fix bug for self._session_pool (#7834)
[0.8.33] - 2023-09-25
New Features
* add pairwise evaluator + benchmark auto-merging retriever (#7810)
Bug Fixes / Nits
* Minor cleanup in embedding class (#7813)
* Misc updates to "OpenAIEmbedding" (#7811)
[0.8.32] - 2023-09-24
New Features
* Added native support for "HuggingFaceEmbedding",
  "InstructorEmbedding", and "OptimumEmbedding" (#7795)
* Added metadata filtering and hybrid search to MyScale vector store
  (#7780)
* Allowing custom text field name for Milvus (#7790)
* Add support for "vector_store_query_mode" to
  "VectorIndexAutoRetriever" (#7797)
Bug Fixes / Nits
* Update "LanceDBVectorStore" to handle score and distance (#7754)
* Pass LLM to "memory_cls" in "CondenseQuestionChatEngine" (#7785)
[0.8.31] - 2023-09-22
New Features
* add pydantic metadata extractor (#7778)
* Allow users to set the embedding dimensions in azure cognitive
  vector store (#7734)
* Add semantic similarity evaluator (#7770)
Bug Fixes / Nits
* 📝docs: Update Chatbot Tutorial and Notebook (#7767)
* Fixed response synthesizers with empty nodes (#7773)
* Fix "NotImplementedError" in auto vector retriever (#7764)
* Multiple kwargs values in "KnowledgeGraphQueryEngine" bug-fix
  (#7763)
* Allow setting azure cognitive search dimensionality (#7734)
* Pass service context to index for dataset generator (#7748)
* Fix output parsers for selector templates (#7774)
* Update Chatbot_SEC.ipynb (#7711)
* linter/typechecker-friendly improvements to cassandra test (#7771)
* Expose debug option of "PgVectorStore" (#7776)
* llms/openai: fix Azure OpenAI by considering "prompt_filter_results"
  field (#7755)
[0.8.30] - 2023-09-21
New Features
* Add support for "gpt-3.5-turbo-instruct" (#7729)
* Add support for "TimescaleVectorStore" (#7727)
* Added "LongContextReorder" for lost-in-the-middle issues (#7719)

ChangeLog

* Add retrieval evals (#7738)
Bug Fixes / Nits
* Added node post-processors to async context chat engine (#7731)
* Added unique index name for postgres tsv column (#7741)
[0.8.29.post1] - 2023-09-18
Bug Fixes / Nits
* Fix langchain import error for embeddings (#7714)
[0.8.29] - 2023-09-18
New Features
* Added metadata filtering to the base simple vector store (#7564)
* add low-level router guide (#7708)
* Add CustomQueryEngine class (#7703)
Bug Fixes / Nits
* Fix context window metadata in lite-llm (#7696)
[0.8.28] - 2023-09-16
New Features
* Add CorrectnessEvaluator (#7661)
* Added support for "Ollama" LLMs (#7635)
* Added "HWPReader" (#7672)
* Simplified portkey LLM interface (#7669)
* Added async operation support to "ElasticsearchStore" vector store
  (#7613)
* Added support for "LiteLLM" (#7600)
* Added batch evaluation runner (#7692)
Bug Fixes / Nits
* Avoid "NotImplementedError" for async langchain embeddings (#7668)
* Imrpoved reliability of LLM selectors (#7678)
* Fixed "query_wrapper_prompt" and "system_prompt" for output parsers
  and completion models (#7678)
* Fixed node attribute inheritance in citation query engine (#7675)
Breaking Changes
* Refactor and update "BaseEvaluator" interface to be more consistent
  (#7661)
  * Use "evaluate" function for generic input
  * Use "evaluate_response" function with "Response" objects from
    llama index query engine
* Update existing evaluators with more explicit naming
  * "ResponseEvaluator" -> "FaithfulnessEvaluator"
  * "QueryResponseEvaluator" -> "RelevancyEvaluator"
  * old names are kept as class aliases for backwards compatibility
[0.8.27] - 2023-09-14
New Features
* add low-level tutorial section (#7673)
Bug Fixes / Nits
* default delta should be a dict (#7665)
* better query wrapper logic on LLMPredictor (#7667)
[0.8.26] - 2023-09-12
New Features
* add non-linear embedding adapter (#7658)
* Add "finetune + RAG" evaluation to knowledge fine-tuning notebook
  (#7643)
Bug Fixes / Nits
* Fixed chunk-overlap for sentence splitter (#7590)
[0.8.25] - 2023-09-12
New Features
* Added "AGENT_STEP" callback event type (#7652)
Bug Fixes / Nits
* Allowed "simple" mode to work with "as_chat_engine()" (#7637)
* Fixed index error in azure streaming (#7646)
* Removed "pdb" from llama-cpp (#7651)
[0.8.24] - 2023-09-11
New Features
* guide: fine-tuning to memorize knowledge (#7626)
* added ability to customize prompt template for eval modules (#7626)
Bug Fixes
* Properly detect "llama-cpp-python" version for loading the default
  GGML or GGUF "llama2-chat-13b" model (#7616)
* Pass in "summary_template" properly with
  "RetrieverQueryEngine.from_args()" (#7621)
* Fix span types in wandb callback (#7631)
[0.8.23] - 2023-09-09
Bug Fixes
* Make sure context and system prompt is included in prompt for first

ChangeLog

  chat for llama2 (#7597)
* Avoid negative chunk size error in refine process (#7607)
* Fix relationships for small documents in hierarchical node parser
  (#7611)
* Update Anyscale Endpoints integration with full streaming and async
  support (#7602)
* Better support of passing credentials as LLM constructor args in
  "OpenAI", "AzureOpenAI", and "Anyscale" (#7602)
Breaking Changes
* Update milvus vector store to support filters and dynamic schemas
  (#7286)
  * See the updated notebook for usage
* Added NLTK to core dependencies to support the default sentence
  splitter (#7606)
[0.8.22] - 2023-09-07
New Features
* Added support for ElasticSearch Vector Store (#7543)
Bug Fixes / Nits
* Fixed small "_index" bug in "ElasticSearchReader" (#7570)
* Fixed bug with prompt helper settings in global service contexts
  (#7576)
* Remove newlines from openai embeddings again (#7588)
* Fixed small bug with setting "query_wrapper_prompt" in the service
  context (#7585)
Breaking/Deprecated API Changes
* Clean up vector store interface to use "BaseNode" instead of
  "NodeWithEmbedding"
  * For majority of users, this is a no-op change
  * For users directly operating with the "VectorStore" abstraction
    and manually constructing "NodeWithEmbedding" objects, this is a
    minor breaking change. Use "TextNode" with "embedding" set
    directly, instead of "NodeWithEmbedding".
[0.8.21] - 2023-09-06
New Features
* add embedding adapter fine-tuning engine + guide (#7565)
* Added support for Azure Cognitive Search vector store (#7469)
* Support delete in supabase (#6951)
* Added support for Espilla vector store (#7539)
* Added support for AnyScale LLM (#7497)
Bug Fixes / Nits
* Default to user-configurable top-k in "VectorIndexAutoRetriever"
  (#7556)
* Catch validation errors for structured responses (#7523)
* Fix streaming refine template (#7561)
[0.8.20] - 2023-09-04
New Features
* Added Portkey LLM integration (#7508)
* Support postgres/pgvector hybrid search (#7501)
* upgrade recursive retriever node reference notebook (#7537)
[0.8.19] - 2023-09-03
New Features
* replace list index with summary index (#7478)
* rename list index to summary index part 2 (#7531)
[0.8.18] - 2023-09-03
New Features
* add agent finetuning guide (#7526)
[0.8.17] - 2023-09-02
New Features
* Make (some) loaders serializable (#7498)
* add node references to recursive retrieval (#7522)
Bug Fixes / Nits
* Raise informative error when metadata is too large during splitting
  (#7513)
* Allow langchain splitter in simple node parser (#7517)
[0.8.16] - 2023-09-01
Bug Fixes / Nits
* fix link to Marvin notebook in docs (#7504)
* Ensure metadata is not "None" in "SimpleWebPageReader" (#7499)
* Fixed KGIndex visualization (#7493)
* Improved empty response in KG Index (#7493)
[0.8.15] - 2023-08-31
New Features
* Added support for "MarvinEntityExtractor" metadata extractor (#7438)
* Added a url_metadata callback to SimpleWebPageReader (#7445)
* Expanded callback logging events (#7472)

ChangeLog

Bug Fixes / Nits
* Only convert newlines to spaces for text 001 embedding models in
  OpenAI (#7484)
* Fix "KnowledgeGraphRagRetriever" for non-nebula indexes (#7488)
* Support defined embedding dimension in "PGVectorStore" (#7491)
* Greatly improved similarity calculation speed for the base vector
  store (#7494)
[0.8.14] - 2023-08-30
New Features
* feat: non-kg heterogeneous graph support in Graph RAG (#7459)
* rag guide (#7480)
Bug Fixes / Nits
* Improve openai fine-tuned model parsing (#7474)
* doing some code de-duplication (#7468)
* support both str and templates for query_wrapper_prompt in HF LLMs
  (#7473)
[0.8.13] - 2023-08-29
New Features
* Add embedding finetuning (#7452)
* Added support for RunGPT LLM (#7401)
* Integration guide and notebook with DeepEval (#7425)
* Added "VectorIndex" and "VectaraRetriever" as a managed index
  (#7440)
* Added support for "to_tool_list" to detect and use async functions
  (#7282)
[0.8.12] - 2023-08-28
New Features
* add openai finetuning class (#7442)
* Service Context to/from dict (#7395)
* add finetuning guide (#7429)
Smaller Features / Nits / Bug Fixes
* Add example how to run FalkorDB docker (#7441)
* Update root.md to use get_response_synthesizer expected type.
  (#7437)
* Bugfix MonsterAPI Pydantic version v2/v1 support. Doc Update (#7432)
[0.8.11.post3] - 2023-08-27
New Features
* AutoMergingRetriever (#7420)
[0.8.10.post1] - 2023-08-25
New Features
* Added support for "MonsterLLM" using MonsterAPI (#7343)
* Support comments fields in NebulaGraphStore and int type VID (#7402)
* Added configurable endpoint for DynamoDB (#6777)
* Add structured answer filtering for Refine response synthesizer
  (#7317)
Bug Fixes / Nits
* Use "utf-8" for json file reader (#7390)
* Fix entity extractor initialization (#7407)
[0.8.9] - 2023-08-24
New Features
* Added support for FalkorDB/RedisGraph graph store (#7346)
* Added directed sub-graph RAG (#7378)
* Added support for "BM25Retriever" (#7342)
Bug Fixes / Nits
* Added "max_tokens" to "Xinference" LLM (#7372)
* Support cache dir creation in multithreaded apps (#7365)
* Ensure temperature is a float for openai (#7382)
* Remove duplicate subjects in knowledge graph retriever (#7378)
* Added support for both pydantic v1 and v2 to allow other apps to
  move forward (#7394)
Breaking/Deprecated API Changes
* Refactor prompt template (#7319)
  * Use "BasePromptTemplate" for generic typing
  * Use "PromptTemplate", "ChatPromptTemplate",
    "SelectorPromptTemplate" as core implementations
  * Use "LangchainPromptTemplate" for compatibility with Langchain
    prompt templates
  * Fully replace specific prompt classes (e.g. "SummaryPrompt") with
    generic "BasePromptTemplate" for typing in codebase.
  * Keep "Prompt" as an alias for "PromptTemplate" for backwards

ChangeLog

    compatibility.
  * BREAKING CHANGE: remove support for
    "Prompt.from_langchain_prompt", please use
    "template=LangchainPromptTemplate(lc_template)" instead.
[0.8.8] - 2023-08-23
New Features
* "OpenAIFineTuningHandler" for collecting LLM inputs/outputs for
  OpenAI fine tuning (#7367)
Bug Fixes / Nits
* Add support for "claude-instant-1.2" (#7369)
[0.8.7] - 2023-08-22
New Features
* Support fine-tuned OpenAI models (#7364)
* Added support for Cassandra vector store (#6784)
* Support pydantic fields in tool functions (#7348)
Bug Fixes / Nits
* Fix infinite looping with forced function call in "OpenAIAgent"
  (#7363)
[0.8.6] - 2023-08-22
New Features
* auto vs. recursive retriever notebook (#7353)
* Reader and Vector Store for BagelDB with example notebooks (#7311)
Bug Fixes / Nits
* Use service context for intermediate index in retry source query
  engine (#7341)
* temp fix for prompt helper + chat models (#7350)
* Properly skip unit-tests when packages not installed (#7351)
[0.8.5.post2] - 2023-08-20
New Features
* Added FireStore docstore/index store support (#7305)
* add recursive agent notebook (#7330)
Bug Fixes / Nits
* Fix Azure pydantic error (#7329)
* fix callback trace ids (make them a context var) (#7331)
[0.8.5.post1] - 2023-08-18
New Features
* Awadb Vector Store (#7291)
Bug Fixes / Nits
* Fix bug in OpenAI llm temperature type
[0.8.5] - 2023-08-18
New Features
* Expose a system prompt/query wrapper prompt in the service context
  for open-source LLMs (#6647)
* Changed default MyScale index format to "MSTG" (#7288)
* Added tracing to chat engines/agents (#7304)
* move LLM and embeddings to pydantic (#7289)
Bug Fixes / Nits
* Fix sentence splitter bug (#7303)
* Fix sentence splitter infinite loop (#7295)
[0.8.4] - 2023-08-17
Bug Fixes / Nits
* Improve SQL Query parsing (#7283)
* Fix loading embed_model from global service context (#7284)
* Limit langchain version until we migrate to pydantic v2 (#7297)
[0.8.3] - 2023-08-16
New Features
* Added Knowledge Graph RAG Retriever (#7204)
Bug Fixes / Nits
* accept "api_key" kwarg in OpenAI LLM class constructor (#7263)
* Fix to create separate queue instances for separate instances of
  "StreamingAgentChatResponse" (#7264)
[0.8.2.post1] - 2023-08-14
New Features
* Added support for Rockset as a vector store (#7111)
Bug Fixes
* Fixed bug in service context definition that could disable LLM
  (#7261)
[0.8.2] - 2023-08-14
New Features
* Enable the LLM or embedding model to be disabled by setting to
  "None" in the service context (#7255)
* Resolve nearly any huggingface embedding model using the
  "embed_model="local:<model_name>"" syntax (#7255)
* Async tool-calling support (#7239)

ChangeLog

Bug Fixes / Nits
* Updated supabase kwargs for add and query (#7103)
* Small tweak to default prompts to allow for more general purpose
  queries (#7254)
* Make callback manager optional for "CustomLLM" + docs update (#7257)
[0.8.1] - 2023-08-13
New Features
* feat: add node_postprocessors to ContextChatEngine (#7232)
* add ensemble query engine tutorial (#7247)
Smaller Features
* Allow EMPTY keys for Fastchat/local OpenAI API endpoints (#7224)
[0.8.0] - 2023-08-11
New Features
* Added "LLAMA_INDEX_CACHE_DIR" to control cached files (#7233)
* Default to pydantic selectors when possible (#7154, #7223)
* Remove the need for langchain wrappers on "embed_model" in the
  service context (#7157)
* Metadata extractors take an "LLM" object now, in addition to
  "LLMPredictor" (#7202)
* Added local mode + fallback to llama.cpp + llama2 (#7200)
* Added local fallback for embeddings to "BAAI/bge-small-en" (#7200)
* Added "SentenceWindowNodeParser" +
  "MetadataReplacementPostProcessor" (#7211)
Breaking Changes
* Change default LLM to gpt-3.5-turbo from text-davinci-003 (#7223)
* Change prompts for compact/refine/tree_summarize to work better with
  gpt-3.5-turbo (#7150, #7179, #7223)
* Increase default LLM temperature to 0.1 (#7180)
[0.7.24.post1] - 2023-08-11
Other Changes
* Reverted #7223 changes to defaults (#7235)
[0.7.24] - 2023-08-10
New Features
* Default to pydantic selectors when possible (#7154, #7223)
* Remove the need for langchain wrappers on "embed_model" in the
  service context (#7157)
* Metadata extractors take an "LLM" object now, in addition to
  "LLMPredictor" (#7202)
* Added local mode + fallback to llama.cpp + llama2 (#7200)
* Added local fallback for embeddings to "BAAI/bge-small-en" (#7200)
* Added "SentenceWindowNodeParser" +
  "MetadataReplacementPostProcessor" (#7211)
Breaking Changes
* Change default LLM to gpt-3.5-turbo from text-davinci-003 (#7223)
* Change prompts for compact/refine/tree_summarize to work better with
  gpt-3.5-turbo (#7150, #7179, #7223)
* Increase default LLM temperature to 0.1 (#7180)
Other Changes
* docs: Improvements to Mendable Search (#7220)
* Refactor openai agent (#7077)
Bug Fixes / Nits
* Use "1 - cosine_distance" for pgvector/postgres vector db (#7217)
* fix metadata formatting and extraction (#7216)
* fix(readers): Fix non-ASCII JSON Reader bug (#7086)
* Chore: change PgVectorStore variable name from "sim" to "distance"
  for clarity (#7226)
[0.7.23] - 2023-08-10
Bug Fixes / Nits
* Fixed metadata formatting with custom tempalates and inheritance
  (#7216)
[0.7.23] - 2023-08-10
New Features
* Add "one click observability" page to docs (#7183)
* Added Xorbits inference for local deployments (#7151)

ChangeLog

* Added Zep vector store integration (#7203)
* feat/zep vectorstore (#7203)
Bug Fixes / Nits
* Update the default "EntityExtractor" model (#7209)
* Make "ChatMemoryBuffer" pickleable (#7205)
* Refactored "BaseOpenAIAgent" (#7077)
[0.7.22] - 2023-08-08
New Features
* add ensemble retriever notebook (#7190)
* DOCS: added local llama2 notebook (#7146)
Bug Fixes / Nits
* Fix for "AttributeError: 'OpenAIAgent' object has no attribute
  'callback_manager'" by calling super constructor within
  "BaseOpenAIAgent"
* Remove backticks from nebula queries (#7192)
[0.7.21] - 2023-08-07
New Features
* Added an "EntityExtractor" for metadata extraction (#7163)
[0.7.20] - 2023-08-06
New Features
* add router module docs (#7171)
* add retriever router (#7166)
New Features
* Added a "RouterRetriever" for routing queries to specific retrievers
  (#7166)
Bug Fixes / Nits
* Fix for issue where having multiple concurrent streamed responses
  from "OpenAIAgent" would result in interleaving of tokens across
  each response stream. (#7164)
* fix llms callbacks issue (args[0] error) (#7165)
[0.7.19] - 2023-08-04
New Features
* Added metadata filtering to weaviate (#7130)
* Added token counting (and all callbacks) to agents and streaming
  (#7122)
[0.7.18] - 2023-08-03
New Features
* Added "to/from_string" and "to/from_dict" methods to memory objects
  (#7128)
* Include columns comments from db tables in table info for SQL
  queries (#7124)
* Add Neo4j support (#7122)
Bug Fixes / Nits
* Added "Azure AD" validation support to the "AzureOpenAI" class
  (#7127)
* add "flush=True" when printing agent/chat engine response stream
  (#7129)
* Added "Azure AD" support to the "AzureOpenAI" class (#7127)
* Update LLM question generator prompt to mention JSON markdown
  (#7105)
* Fixed "astream_chat" in chat engines (#7139)
[0.7.17] - 2023-08-02
New Features
* Update "ReActAgent" to support memory modules (minor breaking change
  since the constructor takes "memory" instead of "chat_history", but
  the main "from_tools" method remains backward compatible.) (#7116)
* Update "ReActAgent" to support streaming (#7119)
* Added Neo4j graph store and query engine integrations (#7122)
* add object streaming (#7117)
[0.7.16] - 2023-07-30
New Features
* Chat source nodes (#7078)
[0.7.15] - 2023-07-29
Bug Fixes / Nits
* anthropic api key customization (#7082)
* Fix broken link to API reference in Contributor Docs (#7080)
* Update vector store docs (#7076)
* Update comment (#7073)
[0.7.14] - 2023-07-28
New Features
* Added HotpotQADistractor benchmark evaluator (#7034)
* Add metadata filter and delete support for LanceDB (#7048)
* Use MetadataFilters in opensearch (#7005)
* Added support for "KuzuGraphStore" (#6970)

ChangeLog

* Added "kg_triplet_extract_fn" to customize how KGs are built (#7068)
Bug Fixes / Nits
* Fix string formatting in context chat engine (#7050)
* Fixed tracing for async events (#7052)
* Less strict triplet extraction for KGs (#7059)
* Add configurable limit to KG data retrieved (#7059)
* Nebula connection improvements (#7059)
* Bug fix in building source nodes for agent response (#7067)
[0.7.13] - 2023-07-26
New Features
* Support function calling api for AzureOpenAI (#7041)
Bug Fixes / Nits
* tune prompt to get rid of KeyError in SubQ engine (#7039)
* Fix validation of Azure OpenAI keys (#7042)
[0.7.12] - 2023-07-25
New Features
* Added "kwargs" to "ComposableGraph" for the underlying query engines
  (#6990)
* Validate openai key on init (#6940)
* Added async embeddings and async RetrieverQueryEngine (#6587)
* Added async "aquery" and "async_add" to PGVectorStore (#7031)
* Added ".source_nodes" attribute to chat engine and agent responses
  (#7029)
* Added "OpenInferenceCallback" for storing generation data in
  OpenInference format (#6998)
Bug Fixes / Nits
* Fix achat memory initialization for data agents (#7000)
* Add "print_response_stream()" to agengt/chat engine response class
  (#7018)
Bug Fixes / Nits
* Fix achat memory initialization for data agents (#7000)
* Add "print_response_stream()" to agengt/chat engine response class
  (#7018)
[v0.7.11.post1] - 2023-07-20
New Features
* Default to pydantic question generation when possible for sub-
  question query engine (#6979)
Bug Fixes / Nits
* Fix returned order of messages in large chat memory (#6979)
[v0.7.11] - 2023-07-19
New Features
* Added a "SentenceTransformerRerank" node post-processor for fast
  local re-ranking (#6934)
* Add numpy support for evaluating queries in pandas query engine
  (#6935)
* Add metadata filtering support for Postgres Vector Storage
  integration (#6968)
* Proper llama2 support for agents and query engines (#6969)
Bug Fixes / Nits
* Added "model_name" to LLMMetadata (#6911)
* Fallback to retriever service context in query engines (#6911)
* Fixed "as_chat_engine()" ValueError with extra kwargs (#6971
[v0.7.10.post1] - 2023-07-18
New Features
* Add support for Replicate LLM (vicuna & llama 2!)
Bug Fixes / Nits
* fix streaming for condense chat engine (#6958)
[v0.7.10] - 2023-07-17
New Features
* Add support for chroma v0.4.0 (#6937)
* Log embedding vectors to callback manager (#6962)
Bug Fixes / Nits
* add more robust embedding timeouts (#6779)
* improved connection session management on postgres vector store
  (#6843)
[v0.7.9] - 2023-07-15
New Features
* specify "embed_model="local"" to use default local embbeddings in
  the service context (#6806)
* Add async "acall" endpoint to "BasePydanticProgram" (defaults to
  sync version). Implement for "OpenAIPydanticProgram"
Bug Fixes / Nits
* fix null metadata for searching existing vector dbs (#6912)

ChangeLog

* add module guide docs for "SimpleDirectoryReader" (#6916)
* make sure "CondenseQuestionChatEngine" streaming chat endpoints work
  even if not explicitly setting "streaming=True" in the underlying
  query engine.
[v0.7.8] - 2023-07-13
New Features
* Added embedding speed benchmark (#6876)
* Added BEIR retrieval benchmark (#6825)
Bug Fixes / Nits
* remove toctrees from deprecated_terms (#6895)
* Relax typing dependencies (#6879)
* docs: modification to evaluation notebook (#6840)
* raise error if the model does not support functions (#6896)
* fix(bench embeddings): bug not taking into account string length
  (#6899)x
[v0.7.7] - 2023-07-13
New Features
* Improved milvus consistency support and output fields support
  (#6452)
* Added support for knowledge graph querying w/ cypyer+nebula (#6642)
* Added "Document.example()" to create documents for fast prototyping
  (#6739)
* Replace react chat engine to use native reactive agent (#6870)
Bug Fixes / Nits
* chore: added a help message to makefile (#6861)
Bug Fixes / Nits
* Fixed support for using SQLTableSchema context_str attribute (#6891)
[v0.7.6] - 2023-07-12
New Features
* Added sources to agent/chat engine responses (#6854)
* Added basic chat buffer memory to agents / chat engines (#6857)
* Adding load and search tool (#6871)
* Add simple agent benchmark (#6869)
* add agent docs (#6866)
* add react agent (#6865)
Breaking/Deprecated API Changes
* Replace react chat engine with native react agent (#6870)
* Set default chat mode to "best": use openai agent when possible,
  otherwise use react agent (#6870)
Bug Fixes / Nits
* Fixed support for legacy vector store metadata (#6867)
* fix chroma notebook in docs (#6872)
* update LC embeddings docs (#6868)
[v0.7.5] - 2023-07-11
New Features
* Add "Anthropic" LLM implementation (#6855)
Bug Fixes / Nits
* Fix indexing error in "SentenceEmbeddingOptimizer" (#6850)
* fix doc for custom embedding model (#6851)
* fix(silent error): Add validation to "SimpleDirectoryReader" (#6819)
* Fix link in docs (#6833)
* Fixes Azure gpt-35-turbo model not recognized (#6828)
* Update Chatbot_SEC.ipynb (#6808)
* Rename leftover original name to LlamaIndex (#6792)
* patch nested traces of the same type (#6791)
[v0.7.4] - 2023-07-08
New Features
* "MetadataExtractor" - Documnent Metadata Augmentation via LLM-based
  feature extractors (#6764)
Bug Fixes / Nits
* fixed passing in query bundle to node postprocessors (#6780)
* fixed error in callback manager with nested traces (#6791)
[v0.7.3] - 2023-07-07
New Features
* Sub question query engine returns source nodes of sub questions in
  the callback manager (#6745)
* trulens integration (#6741)
* Add sources to subquestion engine (#6745)
Bug Fixes / Nits
* Added/Fixed streaming support to simple and condense chat engines
  (#6717)
* fixed "response_mode="no_text"" response synthesizer (#6755)
* fixed error setting "num_output" and "context_window" in service
  context (#6766)
* Fix missing as_query_engine() in tutorial (#6747)

ChangeLog

* Fixed variable sql_query_engine in the notebook (#6778)
* fix required function fields (#6761)
* Remove usage of stop token in Prompt, SQL gen (#6782)
[v0.7.2] - 2023-07-06
New Features
* Support Azure OpenAI (#6718)
* Support prefix messages (e.g. system prompt) in chat engine and
  OpenAI agent (#6723)
* Added "CBEventType.SUB_QUESTIONS" event type for tracking sub
  question queries/responses (#6716)
Bug Fixes / Nits
* Fix HF LLM output error (#6737)
* Add system message support for langchain message templates (#6743)
* Fixed applying node-postprocessors (#6749)
* Add missing "CustomLLM" import under "llama_index.llms" (#6752)
* fix(typo): "get_transformer_tokenizer_fn" (#6729)
* feat(formatting): "black[jupyter]" (#6732)
* fix(test): "test_optimizer_chinese" (#6730)
[v0.7.1] - 2023-07-05
New Features
* Streaming support for OpenAI agents (#6694)
* add recursive retriever + notebook example (#6682)
[v0.7.0] - 2023-07-04
New Features
* Index creation progress bars (#6583)
Bug Fixes/ Nits
* Improved chat refine template (#6645)
Breaking/Deprecated API Changes
* Change "BaseOpenAIAgent" to use "llama_index.llms.OpenAI". Adjust
  "chat_history" to use "List[ChatMessage]]" as type.
* Remove (previously deprecated)
  "llama_index.langchain_helpers.chain_wrapper" module.
* Remove (previously deprecated)
  "llama_index.token_counter.token_counter" module. See migration
  guide for more details on new callback based token counting.
* Remove "ChatGPTLLMPredictor" and "HuggingFaceLLMPredictor". See
  migration guide for more details on replacements.
* Remove support for setting "cache" via "LLMPredictor" constructor.
* Update "BaseChatEngine" interface:
  * adjust "chat_history" to use "List[ChatMessage]]" as type
  * expose "chat_history" state as a property
  * support overriding "chat_history" in "chat" and "achat" endpoints
* Remove deprecated arguments for "PromptHelper": "max_input_size",
  "embedding_limit", "max_chunk_overlap"
* Update all notebooks to use native openai integration (#6696)
[v0.6.38] - 2023-07-02
New Features
* add optional tqdm progress during index creation (#6583)
* Added async support for "compact" and "refine" response modes
  (#6590)
* [feature]add transformer tokenize functionalities for optimizer
  (chinese) (#6659)
* Add simple benchmark for vector store (#6670)
* Introduce "llama_index.llms" module, with new "LLM" interface, and
  "OpenAI", "HuggingFaceLLM", "LangChainLLM" implementations. (#6615)
* Evaporate pydantic program (#6666)
Bug Fixes / Nits
* Improve metadata/node storage and retrieval for RedisVectorStore
  (#6678)
* Fixed node vs. document filtering in vector stores (#6677)
* add context retrieval agent notebook link to docs (#6660)
* Allow null values for the 'image' property in the ImageNode class
  and se… (#6661)
* Fix broken links in docs (#6669)
* update milvus to store node content (#6667)
[v0.6.37] - 2023-06-30

ChangeLog

New Features
* add context augmented openai agent (#6655)
[v0.6.36] - 2023-06-29
New Features
* Redis support for index stores and docstores (#6575)
* DuckDB + SQL query engine notebook (#6628)
* add notebook showcasing deplot data loader (#6638)
Bug Fixes / Nits
* More robust JSON parsing from LLM for "SelectionOutputParser"
  (#6610)
* bring our loaders back in line with llama-hub (#6630)
* Remove usage of SQLStructStoreIndex in notebooks (#6585)
* MD reader: remove html tags and leave linebreaks alone (#6618)
* bump min langchain version to latest version (#6632)
* Fix metadata column name in postgres vector store (#6622)
* Postgres metadata fixes (#6626, #6634)
* fixed links to dataloaders in contribution.md (#6636)
* fix: typo in docs in creating custom_llm huggingface example (#6639)
* Updated SelectionOutputParser to handle JSON objects and arrays
  (#6610)
* Fixed docstring argument typo (#6652)
[v0.6.35] - 2023-06-28
* refactor structured output + pydantic programs (#6604)
Bug Fixes / Nits
* Fix serialization for OpenSearch vector stores (#6612)
* patch docs relationships (#6606)
* Bug fix for ignoring directories while parsing git repo (#4196)
* updated Chroma notebook (#6572)
* Backport old node name (#6614)
* Add the ability to change chroma implementation (#6601)
[v0.6.34] - 2023-06-26
Patch Update (v0.6.34.post1)
* Patch imports for Document obj for backwards compatibility (#6597)
New Features
* New "TextNode"/"Document" object classes based on pydantic (#6586)
* "TextNode"/"Document" objects support metadata customization
  (metadata templates, exclude metadata from LLM or embeddings)
  (#6586)
* Nodes no longer require flat metadata dictionaries, unless the
  vector store you use requires it (#6586)
Bug Fixes / Nits
* use "NLTK_DATA" env var to control NLTK download location (#6579)
* [discord] save author as metadata in group_conversations.py (#6592)
* bs4 -> beautifulsoup4 in requirements (#6582)
* negate euclidean distance (#6564)
* add df output parser notebook link to docs (#6581)
Breaking/Deprecated API Changes
* "Node" has been renamed to "TextNode" and is imported from
  "llama_index.schema" (#6586)
* "TextNode" and "Document" must be instantiated with kwargs:
  "Document(text=text)" (#6586)
* "TextNode" (fka "Node") has a "id_" or "node_id" property, rather
  than "doc_id" (#6586)
* "TextNode" and "Document" have a metadata property, which replaces
  the extra_info property (#6586)
* "TextNode" no longer has a "node_info" property (start/end indexes
  are accessed directly with "start/end_char_idx" attributes) (#6586)
[v0.6.33] - 2023-06-25
New Features
* Add typesense vector store (#6561)
* add df output parser (#6576)
Bug Fixes / Nits
* Track langchain dependency via bridge module. (#6573)
[v0.6.32] - 2023-06-23
New Features
* add object index (#6548)
* add SQL Schema Node Mapping + SQLTableRetrieverQueryEngine + obj
  index fixes (#6569)
* sql refactor (NLSQLTableQueryEngine) (#6529)

ChangeLog

Bug Fixes / Nits
* Update vector_stores.md (#6562)
* Minor "BaseResponseBuilder" interface cleanup (#6557)
* Refactor TreeSummarize (#6550)
[v0.6.31] - 2023-06-22
Bug Fixes / Nits
* properly convert weaviate distance to score (#6545)
* refactor tree summarize and fix bug to not truncate context (#6550)
* fix custom KG retrieval notebook nits (#6551)
[v0.6.30] - 2023-06-21
New Features
* multi-selector support in router query engine (#6518)
* pydantic selector support in router query engine using OpenAI
  function calling API (#6518)
* streaming response support in "CondenseQuestionChatEngine" and
  "SimpleChatEngine" (#6524)
* metadata filtering support in "QdrantVectorStore" (#6476)
* add "PGVectorStore" to support postgres with pgvector (#6190)
Bug Fixes / Nits
* better error handling in the mbox reader (#6248)
* Fix blank similarity score when using weaviate (#6512)
* fix for sorted nodes in "PrevNextNodePostprocessor" (#6048)
Breaking/Deprecated API Changes
* Refactor PandasQueryEngine to take in df directly, deprecate
  PandasIndex (#6527)
[v0.6.29] - 2023-06-20
New Features
* query planning tool with OpenAI Function API (#6520)
* docs: example of kg+vector index (#6497)
* Set context window sizes for Cohere and AI21(J2 model) (#6485)
Bug Fixes / Nits
* add default input size for Cohere and AI21 (#6485)
* docs: replace comma with colon in dict object (#6439)
* extra space in prompt and error message update (#6443)
* [Issue 6417] Fix prompt_templates docs page (#6499)
* Rip out monkey patch and update model to context window mapping
  (#6490)
[v0.6.28] - 2023-06-19
New Features
* New OpenAI Agent + Query Engine Cookbook (#6496)
* allow recursive data extraction (pydantic program) (#6503)
Bug Fixes / Nits
* update mongo interface (#6501)
* fixes that we forgot to include for openai pydantic program (#6503)
  (#6504)
* Fix github pics in Airbyte notebook (#6493)
[v0.6.27] - 2023-06-16
New Features
* Add node doc_id filtering to weaviate (#6467)
* New "TokenCountingCallback" to customize and track embedding,
  prompt, and completion token usage (#6440)
* OpenAI Retrieval Function Agent (#6491)
Breaking/Deprecated API Changes
* Deprecated current token tracking (llm predictor and embed model
  will no longer track tokens in the future, please use the
  "TokenCountingCallback" (#6440)
* Add maximal marginal relevance to the Simple Vector Store, which can
  be enabled as a query mode (#6446)
Bug Fixes / Nits
* "as_chat_engine" properly inherits the current service context
  (#6470)
* Use namespace when deleting from pinecone (#6475)
* Fix paths when using fsspec on windows (#3778)
* Fix for using custom file readers in "SimpleDirectoryReader" (#6477)
* Edit MMR Notebook (#6486)
* FLARE fixes (#6484)
[v0.6.26] - 2023-06-14
New Features
* Add OpenAIAgent and tutorial notebook for "build your own agent"
  (#6461)
* Add OpenAIPydanticProgram (#6462)

ChangeLog

Bug Fixes / Nits
* Fix citation engine import (#6456)
[v0.6.25] - 2023-06-13
New Features
* Added FLARE query engine (#6419).
[v0.6.24] - 2023-06-12
New Features
* Added better support for vector store with existing data (e.g. allow
  configurable text key) for Pinecone and Weaviate. (#6393)
* Support batched upsert for Pineone (#6393)
* Added initial guidance integration. Added "GuidancePydanticProgram"
  for generic structured output generation and
  "GuidanceQuestionGenerator" for generating sub-questions in
  "SubQuestionQueryEngine" (#6246).
[v0.6.23] - 2023-06-11
Bug Fixes / Nits
* Remove hardcoded chunk size for citation query engine (#6408)
* Mongo demo improvements (#6406)
* Fix notebook (#6418)
* Cleanup RetryQuery notebook (#6381)
[v0.6.22] - 2023-06-10
New Features
* Added "SQLJoinQueryEngine" (generalization of
  "SQLAutoVectorQueryEngine") (#6265)
* Added support for graph stores under the hood, and initial support
  for Nebula KG. More docs coming soon! (#2581)
* Added guideline evaluator to allow llm to provide feedback based on
  user guidelines (#4664)
* Added support for MongoDB Vector stores to enable Atlas knnbeta
  search (#6379)
* Added new CitationQueryEngine for inline citations of sources in
  response text (#6239)
Bug Fixes
* Fixed bug with "delete_ref_doc" not removing all metadata from the
  docstore (#6192)
* FIxed bug with loading existing QDrantVectorStore (#6230)
Miscellaneous
* Added changelog officially to github repo (#6191)
[v0.6.21] - 2023-06-06
New Features
* SimpleDirectoryReader has new "filename_as_id" flag to automatically
  set the doc_id (useful for "refresh_ref_docs()")
* DocArray vector store integration
* Tair vector store integration
* Weights and Biases callback handler for tracing and versioning
  indexes
* Can initialize indexes directly from a vector store: "index =
  VectorStoreIndex.from_vector_store(vector_store=vector_store)"
Bug Fixes
* Fixed multimodal notebook
* Updated/fixed the SQL tutorial in the docs
Miscellaneous
* Minor docs updates
* Added github pull-requset templates
* Added github issue-forms
[v0.6.20] - 2023-06-04
New Features
* Added new JSONQueryEngine that uses JSON schema to deliver more
  accurate JSON query answers
* Metadata support for redis vector-store
* Added Supabase vector store integration
Bug Fixes
* Fixed typo in text-to-sql prompt
Breaking/Deprecated API Changes
* Removed GPT prefix from indexes (old imports/names are still
  supported though)
Miscellaneous
* Major docs updates, brought important modules to the top level
[v0.6.19] - 2023-06-02
New Features
* Added agent tool abstraction for llama-hub data loaders
Miscellaneous
* Minor doc updates
[v0.6.18] - 2023-06-02
Miscellaneous
* Added "Discover LlamaIndex" video series to the tutorials docs
  section
* Minor docs updates


Privacy and Security

By default, LLamaIndex sends your data to OpenAI for generating
embeddings and natural language responses. However, it is important to
note that this can be configured according to your preferences.
LLamaIndex provides the flexibility to use your own embedding model or
run a large language model locally if desired.
Data Privacy
Regarding data privacy, when using LLamaIndex with OpenAI, the privacy
details and handling of your data are subject to OpenAI's policies.
And each custom service other than OpenAI have their own policies as
well.
Vector stores
LLamaIndex offers modules to connect with other vector stores within
indexes to store embeddings. It is worth noting that each vector store
has its own privacy policies and practices, and LLamaIndex does not
assume responsibility for how they handle or use your data. Also by
default LLamaIndex have a default option to store your embeddings
locally.


LLMs

A large language model (LLM) is a reasoning engine that can complete
text, chat with users, and follow instructions.
LLM Implementations
LLM Implementations
^^^^^^^^^^^^^^^^^^^
* OpenAI
* Azure OpenAI
* HuggingFaceLLM
* LangChainLLM
* Anthropic
* Gradient Base Model
* Gradient Model Adapter
* LiteLLM
* LlamaCPP
* PaLM
* Predibase
* Replicate
* XOrbits Xinference
LLM Interface
class llama_index.llms.base.LLM(*, callback_manager: CallbackManager = None)
   LLM interface.
   abstract async achat(messages: Sequence[ChatMessage], **kwargs: Any) -> ChatResponse
      Async chat endpoint for LLM.
   abstract async acomplete(prompt: str, **kwargs: Any) -> CompletionResponse
      Async completion endpoint for LLM.
   abstract async astream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> AsyncGenerator[ChatResponse, None]
      Async streaming chat endpoint for LLM.
   abstract async astream_complete(prompt: str, **kwargs: Any) -> AsyncGenerator[CompletionResponse, None]
      Async streaming completion endpoint for LLM.
   abstract chat(messages: Sequence[ChatMessage], **kwargs: Any) -> ChatResponse
      Chat endpoint for LLM.
   abstract classmethod class_name() -> str
      Get the class name, used as a unique ID in serialization.
      This provides a key that makes serialization robust against
      actual class name changes.
   abstract complete(prompt: str, **kwargs: Any) -> CompletionResponse
      Completion endpoint for LLM.
   classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) -> Model
      Creates a new model setting __dict__ and __fields_set__ from
      trusted or pre-validated data. Default values are respected, but
      no other validation is performed. Behaves as if *Config.extra =
      'allow'* was set since it adds all passed values
   copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) -> Model
      Duplicate a model, optionally choose which fields to include,
      exclude and change.
      Parameters:
         * **include** -- fields to include in new model
         * **exclude** -- fields to exclude from new model, as with
           values this takes precedence over include
         * **update** -- values to change/add in the new model. Note:
           the data is not validated before creating the new model:
           you should trust this data
         * **deep** -- set to *True* to make a deep copy of the model
      Returns:
         new model instance
   dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) -> DictStrAny
      Generate a dictionary representation of the model, optionally
      specifying which fields to include or exclude.
   json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) -> unicode

LLMs

      Generate a JSON representation of the model, *include* and
      *exclude* arguments as per *dict()*.
      *encoder* is an optional function to supply as *default* to
      json.dumps(), other arguments as per *json.dumps()*.
   abstract property metadata: LLMMetadata
      LLM metadata.
   abstract stream_chat(messages: Sequence[ChatMessage], **kwargs: Any) -> Generator[ChatResponse, None, None]
      Streaming chat endpoint for LLM.
   abstract stream_complete(prompt: str, **kwargs: Any) -> Generator[CompletionResponse, None, None]
      Streaming completion endpoint for LLM.
   classmethod update_forward_refs(**localns: Any) -> None
      Try to update ForwardRefs on fields based on this Model,
      globalns and localns.
Schemas
class llama_index.llms.base.MessageRole(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
   Message role.
   capitalize()
      Return a capitalized version of the string.
      More specifically, make the first character have upper case and
      the rest lower case.
   casefold()
      Return a version of the string suitable for caseless
      comparisons.
   center(width, fillchar=' ', /)
      Return a centered string of length width.
      Padding is done using the specified fill character (default is a
      space).
   count(sub[, start[, end]]) -> int
      Return the number of non-overlapping occurrences of substring
      sub in string S[start:end].  Optional arguments start and end
      are interpreted as in slice notation.
   encode(encoding='utf-8', errors='strict')
      Encode the string using the codec registered for encoding.
      encoding
         The encoding in which to encode the string.
      errors
         The error handling scheme to use for encoding errors. The
         default is 'strict' meaning that encoding errors raise a
         UnicodeEncodeError.  Other possible values are 'ignore',
         'replace' and 'xmlcharrefreplace' as well as any other name
         registered with codecs.register_error that can handle
         UnicodeEncodeErrors.
   endswith(suffix[, start[, end]]) -> bool
      Return True if S ends with the specified suffix, False
      otherwise. With optional start, test S beginning at that
      position. With optional end, stop comparing S at that position.
      suffix can also be a tuple of strings to try.
   expandtabs(tabsize=8)
      Return a copy where all tab characters are expanded using
      spaces.
      If tabsize is not given, a tab size of 8 characters is assumed.
   find(sub[, start[, end]]) -> int
      Return the lowest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Return -1 on failure.
   format(*args, **kwargs) -> str
      Return a formatted version of S, using substitutions from args
      and kwargs. The substitutions are identified by braces ('{' and
      '}').
   format_map(mapping) -> str
      Return a formatted version of S, using substitutions from
      mapping. The substitutions are identified by braces ('{' and
      '}').
   index(sub[, start[, end]]) -> int
      Return the lowest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Raises ValueError when the substring is not found.
   isalnum()
      Return True if the string is an alpha-numeric string, False
      otherwise.
      A string is alpha-numeric if all characters in the string are
      alpha-numeric and there is at least one character in the string.

LLMs

   isalpha()
      Return True if the string is an alphabetic string, False
      otherwise.
      A string is alphabetic if all characters in the string are
      alphabetic and there is at least one character in the string.
   isascii()
      Return True if all characters in the string are ASCII, False
      otherwise.
      ASCII characters have code points in the range U+0000-U+007F.
      Empty string is ASCII too.
   isdecimal()
      Return True if the string is a decimal string, False otherwise.
      A string is a decimal string if all characters in the string are
      decimal and there is at least one character in the string.
   isdigit()
      Return True if the string is a digit string, False otherwise.
      A string is a digit string if all characters in the string are
      digits and there is at least one character in the string.
   isidentifier()
      Return True if the string is a valid Python identifier, False
      otherwise.
      Call keyword.iskeyword(s) to test whether string s is a reserved
      identifier, such as "def" or "class".
   islower()
      Return True if the string is a lowercase string, False
      otherwise.
      A string is lowercase if all cased characters in the string are
      lowercase and there is at least one cased character in the
      string.
   isnumeric()
      Return True if the string is a numeric string, False otherwise.
      A string is numeric if all characters in the string are numeric
      and there is at least one character in the string.
   isprintable()
      Return True if the string is printable, False otherwise.
      A string is printable if all of its characters are considered
      printable in repr() or if it is empty.
   isspace()
      Return True if the string is a whitespace string, False
      otherwise.
      A string is whitespace if all characters in the string are
      whitespace and there is at least one character in the string.
   istitle()
      Return True if the string is a title-cased string, False
      otherwise.
      In a title-cased string, upper- and title-case characters may
      only follow uncased characters and lowercase characters only
      cased ones.
   isupper()
      Return True if the string is an uppercase string, False
      otherwise.
      A string is uppercase if all cased characters in the string are
      uppercase and there is at least one cased character in the
      string.
   join(iterable, /)
      Concatenate any number of strings.
      The string whose method is called is inserted in between each
      given string. The result is returned as a new string.
      Example: '.'.join(['ab', 'pq', 'rs']) -> 'ab.pq.rs'
   ljust(width, fillchar=' ', /)
      Return a left-justified string of length width.
      Padding is done using the specified fill character (default is a
      space).
   lower()
      Return a copy of the string converted to lowercase.
   lstrip(chars=None, /)
      Return a copy of the string with leading whitespace removed.
      If chars is given and not None, remove characters in chars
      instead.
   static maketrans()
      Return a translation table usable for str.translate().
      If there is only one argument, it must be a dictionary mapping
      Unicode ordinals (integers) or characters to Unicode ordinals,
      strings or None. Character keys will be then converted to
      ordinals. If there are two arguments, they must be strings of
      equal length, and in the resulting dictionary, each character in
      x will be mapped to the character at the same position in y. If
      there is a third argument, it must be a string, whose characters

LLMs

      will be mapped to None in the result.
   partition(sep, /)
      Partition the string into three parts using the given separator.
      This will search for the separator in the string.  If the
      separator is found, returns a 3-tuple containing the part before
      the separator, the separator itself, and the part after it.
      If the separator is not found, returns a 3-tuple containing the
      original string and two empty strings.
   removeprefix(prefix, /)
      Return a str with the given prefix string removed if present.
      If the string starts with the prefix string, return
      string[len(prefix):]. Otherwise, return a copy of the original
      string.
   removesuffix(suffix, /)
      Return a str with the given suffix string removed if present.
      If the string ends with the suffix string and that suffix is not
      empty, return string[:-len(suffix)]. Otherwise, return a copy of
      the original string.
   replace(old, new, count=-1, /)
      Return a copy with all occurrences of substring old replaced by
      new.
         count
            Maximum number of occurrences to replace. -1 (the default
            value) means replace all occurrences.
      If the optional argument count is given, only the first count
      occurrences are replaced.
   rfind(sub[, start[, end]]) -> int
      Return the highest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Return -1 on failure.
   rindex(sub[, start[, end]]) -> int
      Return the highest index in S where substring sub is found, such
      that sub is contained within S[start:end].  Optional arguments
      start and end are interpreted as in slice notation.
      Raises ValueError when the substring is not found.
   rjust(width, fillchar=' ', /)
      Return a right-justified string of length width.
      Padding is done using the specified fill character (default is a
      space).
   rpartition(sep, /)
      Partition the string into three parts using the given separator.
      This will search for the separator in the string, starting at
      the end. If the separator is found, returns a 3-tuple containing
      the part before the separator, the separator itself, and the
      part after it.
      If the separator is not found, returns a 3-tuple containing two
      empty strings and the original string.
   rsplit(sep=None, maxsplit=-1)
      Return a list of the substrings in the string, using sep as the
      separator string.
         sep
            The separator used to split the string.
            When set to None (the default value), will split on any
            whitespace character (including n r t f and spaces) and
            will discard empty strings from the result.
         maxsplit
            Maximum number of splits (starting from the left). -1 (the
            default value) means no limit.
      Splitting starts at the end of the string and works to the
      front.
   rstrip(chars=None, /)
      Return a copy of the string with trailing whitespace removed.
      If chars is given and not None, remove characters in chars
      instead.
   split(sep=None, maxsplit=-1)
      Return a list of the substrings in the string, using sep as the
      separator string.
         sep
            The separator used to split the string.
            When set to None (the default value), will split on any
            whitespace character (including n r t f and spaces) and
            will discard empty strings from the result.
         maxsplit
            Maximum number of splits (starting from the left). -1 (the

LLMs

            default value) means no limit.
      Note, str.split() is mainly useful for data that has been
      intentionally delimited.  With natural text that includes
      punctuation, consider using the regular expression module.
   splitlines(keepends=False)
      Return a list of the lines in the string, breaking at line
      boundaries.
      Line breaks are not included in the resulting list unless
      keepends is given and true.
   startswith(prefix[, start[, end]]) -> bool
      Return True if S starts with the specified prefix, False
      otherwise. With optional start, test S beginning at that
      position. With optional end, stop comparing S at that position.
      prefix can also be a tuple of strings to try.
   strip(chars=None, /)
      Return a copy of the string with leading and trailing whitespace
      removed.
      If chars is given and not None, remove characters in chars
      instead.
   swapcase()
      Convert uppercase characters to lowercase and lowercase
      characters to uppercase.
   title()
      Return a version of the string where each word is titlecased.
      More specifically, words start with uppercased characters and
      all remaining cased characters have lower case.
   translate(table, /)
      Replace each character in the string using the given translation
      table.
         table
            Translation table, which must be a mapping of Unicode
            ordinals to Unicode ordinals, strings, or None.
      The table must implement lookup/indexing via __getitem__, for
      instance a dictionary or list.  If this operation raises
      LookupError, the character is left untouched.  Characters mapped
      to None are deleted.
   upper()
      Return a copy of the string converted to uppercase.
   zfill(width, /)
      Pad a numeric string with zeros on the left, to fill a field of
      the given width.
      The string is never truncated.
pydantic model llama_index.llms.base.ChatMessage
   Chat message.
      {
         "title": "ChatMessage",
         "description": "Chat message.",
         "type": "object",
         "properties": {
            "role": {
               "default": "user",
               "allOf": [
                  {
                     "$ref": "#/definitions/MessageRole"
                  }
               ]
            },
            "content": {
               "title": "Content",
               "default": "",
               "type": "string"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "type": "object"
            }
         },
         "definitions": {
            "MessageRole": {
               "title": "MessageRole",
               "description": "Message role.",
               "enum": [
                  "system",
                  "user",
                  "assistant",
                  "function"
               ],
               "type": "string"
            }
         }
      }
   Fields:
      * "additional_kwargs (dict)"
      * "content (Optional[str])"
      * "role (llama_index.llms.base.MessageRole)"
   field additional_kwargs: dict [Optional]
   field content: Optional[str] = ''
   field role: MessageRole = MessageRole.USER
pydantic model llama_index.llms.base.ChatResponse
   Chat response.
      {
         "title": "ChatResponse",
         "description": "Chat response.",
         "type": "object",
         "properties": {
            "message": {
               "$ref": "#/definitions/ChatMessage"
            },
            "raw": {
               "title": "Raw",
               "type": "object"
            },
            "delta": {
               "title": "Delta",
               "type": "string"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "type": "object"
            }
         },
         "required": [
            "message"
         ],

LLMs

         "definitions": {
            "MessageRole": {
               "title": "MessageRole",
               "description": "Message role.",
               "enum": [
                  "system",
                  "user",
                  "assistant",
                  "function"
               ],
               "type": "string"
            },
            "ChatMessage": {
               "title": "ChatMessage",
               "description": "Chat message.",
               "type": "object",
               "properties": {
                  "role": {
                     "default": "user",
                     "allOf": [
                        {
                           "$ref": "#/definitions/MessageRole"
                        }
                     ]
                  },
                  "content": {
                     "title": "Content",
                     "default": "",
                     "type": "string"
                  },
                  "additional_kwargs": {
                     "title": "Additional Kwargs",
                     "type": "object"
                  }
               }
            }
         }
      }
   Fields:
      * "additional_kwargs (dict)"
      * "delta (Optional[str])"
      * "message (llama_index.llms.base.ChatMessage)"
      * "raw (Optional[dict])"
   field additional_kwargs: dict [Optional]
   field delta: Optional[str] = None
   field message: ChatMessage [Required]
   field raw: Optional[dict] = None
pydantic model llama_index.llms.base.CompletionResponse
   Completion response.
   Fields:
      text: Text content of the response if not streaming, or if
      streaming,
         the current extent of streamed text.
      additional_kwargs: Additional information on the response(i.e.
      token
         counts, function calling information).
      raw: Optional raw JSON that was parsed to populate text, if
      relevant. delta: New text that just streamed in (only relevant
      when streaming).
      {
         "title": "CompletionResponse",
         "description": "Completion response.\n\nFields:\n    text: Text content of the response if not streaming, or if streaming,\n        the current extent of streamed text.\n    additional_kwargs: Additional information on the response(i.e. token\n        counts, function calling information).\n    raw: Optional raw JSON that was parsed to populate text, if relevant.\n    delta: New text that just streamed in (only relevant when streaming).",
         "type": "object",
         "properties": {
            "text": {
               "title": "Text",
               "type": "string"
            },
            "additional_kwargs": {
               "title": "Additional Kwargs",
               "type": "object"
            },
            "raw": {
               "title": "Raw",
               "type": "object"
            },
            "delta": {
               "title": "Delta",
               "type": "string"
            }
         },
         "required": [
            "text"
         ]
      }
   Fields:
      * "additional_kwargs (dict)"
      * "delta (Optional[str])"
      * "raw (Optional[dict])"
      * "text (str)"
   field additional_kwargs: dict [Optional]
   field delta: Optional[str] = None
   field raw: Optional[dict] = None
   field text: str [Required]
pydantic model llama_index.llms.base.LLMMetadata
   LLM metadata.
      {
         "title": "LLMMetadata",
         "description": "LLM metadata.",
         "type": "object",
         "properties": {
            "context_window": {
               "title": "Context Window",
               "default": 3900,
               "type": "integer"
            },
            "num_output": {
               "title": "Num Output",
               "default": 256,
               "type": "integer"
            },
            "is_chat_model": {
               "title": "Is Chat Model",
               "default": false,
               "type": "boolean"

LLMs

            },
            "is_function_calling_model": {
               "title": "Is Function Calling Model",
               "default": false,
               "type": "boolean"
            },
            "model_name": {
               "title": "Model Name",
               "default": "unknown",
               "type": "string"
            }
         }
      }
   Fields:
      * "context_window (int)"
      * "is_chat_model (bool)"
      * "is_function_calling_model (bool)"
      * "model_name (str)"
      * "num_output (int)"
   field context_window: int = 3900
   field is_chat_model: bool = False
   field is_function_calling_model: bool = False
   field model_name: str = 'unknown'
   field num_output: int = 256


Response

Response schema.
class llama_index.response.schema.PydanticResponse(response: ~typing.Optional[~pydantic.main.BaseModel], source_nodes: ~typing.List[~llama_index.schema.NodeWithScore] = <factory>, metadata: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None)
   PydanticResponse object.
   Returned if streaming=False.
   response
      The response text.
      Type:
         Optional[pydantic.main.BaseModel]
   get_formatted_sources(length: int = 100) -> str
      Get formatted sources text.
   get_response() -> Response
      Get a standard response object.
class llama_index.response.schema.Response(response: ~typing.Optional[str], source_nodes: ~typing.List[~llama_index.schema.NodeWithScore] = <factory>, metadata: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None)
   Response object.
   Returned if streaming=False.
   response
      The response text.
      Type:
         Optional[str]
   get_formatted_sources(length: int = 100) -> str
      Get formatted sources text.
class llama_index.response.schema.StreamingResponse(response_gen: ~typing.Generator[str, None, None], source_nodes: ~typing.List[~llama_index.schema.NodeWithScore] = <factory>, metadata: ~typing.Optional[~typing.Dict[str, ~typing.Any]] = None, response_txt: ~typing.Optional[str] = None)
   StreamingResponse object.
   Returned if streaming=True.
   response_gen
      The response generator.
      Type:
         Generator[str, None, None]
   get_formatted_sources(length: int = 100, trim_text: int = True) -> str
      Get formatted sources text.
   get_response() -> Response
      Get a standard response object.
   print_response_stream() -> None
      Print the response stream.


Prompt Templates

These are the reference prompt templates.
We first show links to default prompts.
We then show the base prompt template class and its subclasses.
Default Prompts
* Completion prompt templates.
* Chat prompt templates.
* Selector prompt templates.
Prompt Classes
pydantic model llama_index.prompts.base.BasePromptTemplate
      {
         "title": "BasePromptTemplate",
         "type": "object",
         "properties": {
            "metadata": {
               "title": "Metadata",
               "type": "object"
            },
            "template_vars": {
               "title": "Template Vars",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "kwargs": {
               "title": "Kwargs",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "output_parser": {
               "title": "Output Parser"
            }
         },
         "required": [
            "metadata",
            "template_vars",
            "kwargs"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "kwargs (Dict[str, str])"
      * "metadata (Dict[str, Any])"
      * "output_parser (Optional[llama_index.types.BaseOutputParser])"
      * "template_vars (List[str])"
   field kwargs: Dict[str, str] [Required]
   field metadata: Dict[str, Any] [Required]
   field output_parser: Optional[BaseOutputParser] = None
   field template_vars: List[str] [Required]
   abstract format(llm: Optional[LLM] = None, **kwargs: Any) -> str
   abstract format_messages(llm: Optional[LLM] = None, **kwargs: Any) -> List[ChatMessage]
   abstract get_template(llm: Optional[LLM] = None) -> str
   abstract partial_format(**kwargs: Any) -> BasePromptTemplate
pydantic model llama_index.prompts.base.PromptTemplate
      {
         "title": "PromptTemplate",
         "type": "object",
         "properties": {
            "metadata": {
               "title": "Metadata",
               "type": "object"
            },
            "template_vars": {
               "title": "Template Vars",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "kwargs": {
               "title": "Kwargs",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "output_parser": {
               "title": "Output Parser"
            },
            "template": {
               "title": "Template",
               "type": "string"
            }
         },
         "required": [
            "metadata",
            "template_vars",
            "kwargs",
            "template"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "template (str)"
   field template: str [Required]
   format(llm: Optional[LLM] = None, **kwargs: Any) -> str
      Format the prompt into a string.
   format_messages(llm: Optional[LLM] = None, **kwargs: Any) -> List[ChatMessage]
      Format the prompt into a list of chat messages.
   get_template(llm: Optional[LLM] = None) -> str
   partial_format(**kwargs: Any) -> PromptTemplate
      Partially format the prompt.
pydantic model llama_index.prompts.base.ChatPromptTemplate
      {
         "title": "ChatPromptTemplate",
         "type": "object",
         "properties": {
            "metadata": {
               "title": "Metadata",
               "type": "object"

Prompt Templates

            },
            "template_vars": {
               "title": "Template Vars",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "kwargs": {
               "title": "Kwargs",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "output_parser": {
               "title": "Output Parser"
            },
            "message_templates": {
               "title": "Message Templates",
               "type": "array",
               "items": {
                  "$ref": "#/definitions/ChatMessage"
               }
            }
         },
         "required": [
            "metadata",
            "template_vars",
            "kwargs",
            "message_templates"
         ],
         "definitions": {
            "MessageRole": {
               "title": "MessageRole",
               "description": "Message role.",
               "enum": [
                  "system",
                  "user",
                  "assistant",
                  "function"
               ],
               "type": "string"
            },
            "ChatMessage": {
               "title": "ChatMessage",
               "description": "Chat message.",
               "type": "object",
               "properties": {
                  "role": {
                     "default": "user",
                     "allOf": [
                        {
                           "$ref": "#/definitions/MessageRole"
                        }
                     ]
                  },
                  "content": {
                     "title": "Content",
                     "default": "",
                     "type": "string"
                  },
                  "additional_kwargs": {
                     "title": "Additional Kwargs",
                     "type": "object"
                  }
               }
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "message_templates (List[llama_index.llms.base.ChatMessage])"
   field message_templates: List[ChatMessage] [Required]
   format(llm: Optional[LLM] = None, **kwargs: Any) -> str
   format_messages(llm: Optional[LLM] = None, **kwargs: Any) -> List[ChatMessage]
   get_template(llm: Optional[LLM] = None) -> str
   partial_format(**kwargs: Any) -> ChatPromptTemplate
pydantic model llama_index.prompts.base.SelectorPromptTemplate
      {
         "title": "SelectorPromptTemplate",
         "type": "object",
         "properties": {
            "metadata": {
               "title": "Metadata",
               "type": "object"
            },
            "template_vars": {
               "title": "Template Vars",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "kwargs": {
               "title": "Kwargs",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "output_parser": {
               "title": "Output Parser"
            },
            "default_template": {
               "title": "Default Template"
            },
            "conditionals": {
               "title": "Conditionals"
            }
         },
         "required": [
            "metadata",
            "template_vars",
            "kwargs"
         ]
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "conditionals
        (Optional[List[Tuple[Callable[[llama_index.llms.base.LLM],
        bool], llama_index.prompts.base.BasePromptTemplate]]])"
      * "default_template
        (llama_index.prompts.base.BasePromptTemplate)"
   field conditionals: Optional[List[Tuple[Callable[[LLM], bool], BasePromptTemplate]]] = None
   field default_template: BasePromptTemplate [Required]
   format(llm: Optional[LLM] = None, **kwargs: Any) -> str

Prompt Templates

      Format the prompt into a string.
   format_messages(llm: Optional[LLM] = None, **kwargs: Any) -> List[ChatMessage]
      Format the prompt into a list of chat messages.
   get_template(llm: Optional[LLM] = None) -> str
   partial_format(**kwargs: Any) -> SelectorPromptTemplate
pydantic model llama_index.prompts.base.LangchainPromptTemplate
      {
         "title": "LangchainPromptTemplate",
         "type": "object",
         "properties": {
            "metadata": {
               "title": "Metadata",
               "type": "object"
            },
            "template_vars": {
               "title": "Template Vars",
               "type": "array",
               "items": {
                  "type": "string"
               }
            },
            "kwargs": {
               "title": "Kwargs",
               "type": "object",
               "additionalProperties": {
                  "type": "string"
               }
            },
            "output_parser": {
               "title": "Output Parser"
            },
            "selector": {
               "$ref": "#/definitions/ConditionalPromptSelector"
            }
         },
         "required": [
            "metadata",
            "template_vars",
            "kwargs",
            "selector"
         ],
         "definitions": {
            "BaseOutputParser": {
               "title": "BaseOutputParser",
               "description": "Base class to parse the output of an LLM call.\n\nOutput parsers help structure language model responses.\n\nExample:\n    .. code-block:: python\n\n        class BooleanOutputParser(BaseOutputParser[bool]):\n            true_val: str = \"YES\"\n            false_val: str = \"NO\"\n\n            def parse(self, text: str) -> bool:\n                cleaned_text = text.strip().upper()\n                if cleaned_text not in (self.true_val.upper(), self.false_val.upper()):\n                    raise OutputParserException(\n                        f\"BooleanOutputParser expected output value to either be \"\n                        f\"{self.true_val} or {self.false_val} (case-insensitive). \"\n                        f\"Received {cleaned_text}.\"\n                    )\n                return cleaned_text == self.true_val.upper()\n\n                @property\n                def _type(self) -> str:\n                        return \"boolean_output_parser\"",
               "type": "object",
               "properties": {}
            },
            "BasePromptTemplate": {
               "title": "BasePromptTemplate",
               "description": "Base class for all prompt templates, returning a prompt.",
               "type": "object",
               "properties": {
                  "input_variables": {
                     "title": "Input Variables",
                     "type": "array",
                     "items": {
                        "type": "string"
                     }
                  },
                  "input_types": {
                     "title": "Input Types",
                     "type": "object"
                  },
                  "output_parser": {
                     "$ref": "#/definitions/BaseOutputParser"
                  }
               },
               "required": [
                  "input_variables"
               ]
            },
            "ConditionalPromptSelector": {
               "title": "ConditionalPromptSelector",
               "description": "Prompt collection that goes through conditionals.",
               "type": "object",
               "properties": {
                  "default_prompt": {
                     "$ref": "#/definitions/BasePromptTemplate"
                  }
               },
               "required": [
                  "default_prompt"
               ]
            }
         }
      }
   Config:
      * **arbitrary_types_allowed**: *bool = True*
   Fields:
      * "selector
        (langchain.chains.prompt_selector.ConditionalPromptSelector)"
   field selector: ConditionalPromptSelector [Required]
   format(llm: Optional[LLM] = None, **kwargs: Any) -> str
      Format the prompt into a string.

Prompt Templates

   format_messages(llm: Optional[LLM] = None, **kwargs: Any) -> List[ChatMessage]
      Format the prompt into a list of chat messages.
   get_template(llm: Optional[LLM] = None) -> str
   partial_format(**kwargs: Any) -> BasePromptTemplate
      Partially format the prompt.
Subclass Prompts (deprecated)
Deprecated, but still available for reference at this link.