semvec/pypi-packages huggingface dataset:
llama-index-postprocessor-dashscope-rerank 0.4.1
llama-index postprocessor dashscope-rerank integration
Requires Python: <4.0,>=3.9

# LlamaIndex Postprocessor Integration: DashScope-Rerank

The `llama-index-postprocessor-dashscope-rerank` package contains LlamaIndex integrations for the `gte-rerank` series models provided by Alibaba Tongyi Laboratory.

## Installation

```shell
pip install --upgrade llama-index llama-index-core llama-index-postprocessor-dashscope-rerank
```

## Setup

**Get started:**

1. Obtain the **API-KEY** from the [Alibaba Cloud ModelStudio platform](https://help.aliyun.com/document_detail/2712195.html?spm=a2c4g.2587460.0.i6).
2. Set **API-KEY**

```shell
export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY
```

**Example:**

```python
from llama_index.core.data_structs import Node
from llama_index.core.schema import NodeWithScore
from llama_index.postprocessor.dashscope_rerank import DashScopeRerank

nodes = [
    NodeWithScore(node=Node(text="text1"), score=0.7),
    NodeWithScore(node=Node(text="text2"), score=0.8),
]

dashscope_rerank = DashScopeRerank(top_n=5)
results = dashscope_rerank.postprocess_nodes(nodes, query_str="<user query>")
for res in results:
    print("Text: ", res.node.get_content(), "Score: ", res.score)
```

**output**

```text
Text:  text1 Score:  0.25589250620997755
Text:  text2 Score:  0.18071043165292258
```

### Parameters

|       Name       |  Type  |                                                                                                             Description                                                                                                              |   Default    |
| :--------------: | :----: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :----------: |
|      model       | `str`  |                                                                                                              model name                                                                                                              | `gte-rerank` |
|      top_n       | `int`  | The number of top documents to be returned in the ranking; if not specified, all candidate documents will be returned. If the specified top_n value exceeds the number of input candidate documents, all documents will be returned. |     `3`      |
| return_documents | `bool` |                                                    Whether to return the original text for each document in the returned sorted result list, with the default value being False.                                                     |   `False`    |
|     api_key      | `str`  |                                                                                                        The DashScope api key.                                                                                                        |    `None`    |
llama-index-llms-you 0.4.1
llama-index LLM integration with You.com's conversational Smart and Research APIs
Requires Python: <4.0,>=3.9

# LlamaIndex Llms Integration: You

Integration with [You.com's conversational Smart and Research APIs](https://documentation.you.com/api-reference/).

Each API endpoint is designed to generate conversational
responses to a variety of query types, including inline citations
and web results when relevant.

Smart Mode:

- Quick, reliable answers for a variety of questions
- Cites the entire web page URL

Research Mode:

- In-depth answers with extensive citations for a variety of questions
- Cites the specific web page snippet relevant to the claim

To connect to the You.com api requires an API key which
you can get at https://api.you.com.

For more information, check out the documentations at
https://documentation.you.com/api-reference/.
llama-index-readers-myscale 0.4.1
llama-index readers myscale integration
Requires Python: <4.0,>=3.9

# LlamaIndex Readers Integration: Myscale

## Overview

MyScale Reader allows loading data from a MyScale backend. It constructs a query to retrieve documents based on a given query vector and additional search parameters.

### Installation

You can install Myscale Reader via pip:

```bash
pip install llama-index-readers-myscale
```

### Usage

```python
from llama_index.readers.myscale import MyScaleReader

# Initialize MyScaleReader
reader = MyScaleReader(
    myscale_host="<MyScale Host>",  # MyScale host address
    username="<Username>",  # Username to login
    password="<Password>",  # Password to login
    database="<Database Name>",  # Database name (default: 'default')
    table="<Table Name>",  # Table name (default: 'llama_index')
    index_type="<Index Type>",  # Index type (default: "IVFLAT")
    metric="<Metric>",  # Metric to compute distance (default: 'cosine')
    batch_size=32,  # Batch size for inserting documents (default: 32)
    index_params=None,  # Index parameters for MyScale (default: None)
    search_params=None,  # Search parameters for MyScale query (default: None)
)

# Load data from MyScale
documents = reader.load_data(
    query_vector=[0.1, 0.2, 0.3],  # Query vector
    where_str="<Where Condition>",  # Where condition string (default: None)
    limit=10,  # Number of results to return (default: 10)
)
```

This loader is designed to be used as a way to load data into
[LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently
used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent.
llama-index-readers-stripe-docs 0.4.1
llama-index readers stripe_docs integration
Requires Python: <4.0,>=3.9
Keywords: documentation, stripe

# StripeDocs Loader

```bash
pip install llama-index-readers-stripe-docs
```

This loader asynchronously loads data from the [Stripe documentation](https://stripe.com/docs). It iterates through the Stripe sitemap to get all `/docs` references.

It is based on the [Async Website Loader](https://llamahub.ai/l/web-async_web).

## Usage

```python
from llama_index.core import VectorStoreIndex
from llama_index.readers.stripe_docs import StripeDocsReader

loader = StripeDocsReader()
documents = loader.load_data()

index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
query_engine.query("How do I accept payments on my website?")
```

The `StripeDocsReader` allows you to return plain text docs by setting `html_to_text=True`. You can also adjust the maximum concurrent requests by setting `limit=10`.

## Filtering

You can filter pages from the Stripe sitemap by adding the _filters_ argument to the load_data method. This allows you to control what pages from the Stripe website, including documentation, will be loaded.

The default filters are set to `["/docs"]` to scope everything to docs only.

```python
documents = loader.load_data(filters=["/terminal"])
```
llama-index-tools-database 0.4.1
llama-index tools database integration
Requires Python: <4.0,>=3.9
Keywords: aws rds, postgres, snowflake, sql

# Database Tool

This tool connects to a database (using SQLAlchemy under the hood) and allows an Agent to query the database and get information about the tables.

## Usage

This tool has more extensive example usage documented in a Jupyter notebook [here](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/tools/llama-index-tools-database/examples/database.ipynb).

Here's an example usage of the DatabaseToolSpec.

```python
from llama_index.tools.database import DatabaseToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

db_tools = DatabaseToolSpec(
    scheme="postgresql",  # Database Scheme
    host="localhost",  # Database Host
    port="5432",  # Database Port
    user="postgres",  # Database User
    password="FakeExamplePassword",  # Database Password
    dbname="postgres",  # Database Name
)
agent = FunctionAgent(
    tools=db_tools.to_tool_list(),
    llm=OpenAI(model="gpt-4.1"),
)

print(await agent.run("What tables does this database contain"))
print(await agent.run("Describe the first table"))
print(await agent.run("Retrieve the first row of that table"))
```

The tools available are:

`list_tables`: A tool to list the tables in the database schema
`describe_tables`: A tool to describe the schema of a table
`load_data`: A tool that accepts an SQL query and returns the result

This loader is designed to be used as a way to load data as a Tool in a Agent.
llama-index-readers-qdrant 0.4.1
llama-index readers qdrant integration
Requires Python: <4.0,>=3.9

# LlamaIndex Readers Integration: Qdrant

## Overview

The Qdrant Reader allows you to retrieve documents from existing Qdrant collections. Qdrant is a similarity search engine that helps you efficiently search and retrieve similar items from large datasets based on vector embeddings.

For more detailed information about Qdrant, visit [Qdrant](qdrant.io)

### Installation

You can install the Qdrant Reader via pip:

```bash
pip install llama-index-readers-qdrant
```

### Usage

```python
from llama_index.readers.qdrant import QdrantReader

# Initialize QdrantReader
reader = QdrantReader(
    location="<Qdrant Location>",
    url="<Qdrant URL>",
    port="<Port>",
    grpc_port="<gRPC Port>",
    prefer_grpc="<Prefer gRPC>",
    https="<Use HTTPS>",
    api_key="<API Key>",
    prefix="<URL Prefix>",
    timeout="<Timeout>",
    host="<Host>",
)

# Load data from Qdrant
documents = reader.load_data(
    collection_name="<Collection Name>",
    query_vector=[0.1, 0.2, 0.3],
    should_search_mapping={"text_field": "text"},
    must_search_mapping={"text_field": "text"},
    must_not_search_mapping={"text_field": "text"},
    rang_search_mapping={"text_field": {"gte": 0.1, "lte": 0.2}},
    limit=10,
)
```

This loader is designed to be used as a way to load data into
[LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently
used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent.
llama-index-packs-llama-dataset-metadata 0.4.1
llama-index packs llama_dataset_metadata integration
Requires Python: <4.0,>=3.9
Keywords: evaluation, llamadataset, rag, submission

# LlamaDataset Metadata Pack

As part of the `LlamaDataset` submission package into [llamahub](https://llamahub.ai),
two metadata files are required, namely: `card.json` and `README.md`. This pack
creates these two files and saves them to disk to help expedite the submission
process.

## CLI Usage

You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:

```bash
llamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack
```

You can then inspect the files at `./llama_dataset_metadata_pack` and use them as a template for your own project!

## Code Usage

You can download the pack to the `./llama_dataset_metadata_pack` directory through python
code as well. The sample script below demonstrates how to construct `LlamaDatasetMetadataPack`
using a `LabelledRagDataset` downloaded from `llama-hub` and a simple RAG pipeline
built off of its source documents.

```python
from llama_index.core.llama_pack import download_llama_pack

# Download and install dependencies
LlamaDatasetMetadataPack = download_llama_pack(
    "LlamaDatasetMetadataPack", "./llama_dataset_metadata_pack"
)

# construction requires a query_engine, a rag_dataset, and optionally a judge_llm
llama_dataset_metadata_pack = LlamaDatasetMetadataPack()

# create and save `card.json` and `README.md` to disk
dataset_description = (
    "A labelled RAG dataset based off an essay by Paul Graham, consisting of "
    "queries, reference answers, and reference contexts."
)

llama_dataset_metadata_pack.run(
    name="Paul Graham Essay Dataset",
    description=dataset_description,
    rag_dataset=rag_dataset,  # defined earlier not shown here
    index=index,  # defined earlier not shown here
    benchmark_df=benchmark_df,  # defined earlier not shown here
    baseline_name="llamaindex",
)
```

NOTE: this pack should be used only after performing a RAG evaluation (i.e., by
using `RagEvaluatorPack` on a `LabelledRagDataset`). In the code snippet above,
`index`, `rag_dataset`, and `benchmark_df` are all objects that you'd expect to
have only after performing the RAG evaluation as mention in the previous sentence.
llama-index-readers-pathway 0.4.1
llama-index readers pathway integration
Requires Python: <4.0,>=3.9

# LlamaIndex Readers Integration: Pathway

## Overview

Pathway Reader is a utility class for retrieving documents from the Pathway data indexing pipeline. It queries the Pathway vector store to get the closest neighbors of a given text query.

### Installation

You can install Pathway Reader via pip:

```bash
pip install llama-index-readers-pathway
```

### Usage

```python
from llama_index.readers.pathway import PathwayReader

# Initialize PathwayReader with the URI and port of the Pathway server
reader = PathwayReader(host="<Pathway Host>", port="<Port>")

# Load data from Pathway
documents = reader.load_data(
    query_text="<Query Text>",  # The text to get the closest neighbors of
    k=4,  # Number of results to return
    metadata_filter="<Metadata Filter>",  # Filter to be applied
)
```

This loader is designed to be used as a way to load data into
[LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index).
llama-index-llms-netmind 0.2.1
llama-index llms netmind integration
Requires Python: <4.0,>=3.9

# LlamaIndex Llms Integration: Netmind

## Installation

To install the required package, run:

```shell
pip install llama-index-llms-netmind
```

## Setup

1. Set your Netmind API key as an environment variable. Visit https://www.netmind.ai/ and sign up to get an API key.

```shell
import os

os.environ["NETMIND_API_KEY"] = "you_api_key"
```

## Basic Usage

### Generate Completions

```python
from llama_index.llms.netmind import NetmindLLM

llm = NetmindLLM(
    model="meta-llama/Llama-3.3-70B-Instruct", api_key="your api key"
)

resp = llm.complete("Is 9.9 or 9.11 bigger?")
print(resp)
```

### Chat Responses

To send a chat message and receive a response, create a list of `ChatMessage` instances and use the `chat` method:

```python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="What is your name"),
]
resp = llm.chat(messages)
print(resp)
```
llama-index-readers-llama-parse 0.5.1
llama-index readers llama-parse integration
Requires Python: <4.0,>=3.9
Keywords: PDF, llama, llama-parse, parse

# LlamaParse

LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks.

LlamaParse directly integrates with [LlamaIndex](https://github.com/run-llama/llama_index).

Currently available for **free**. Try it out today!

## Getting Started

First, login and get an api-key from `https://cloud.llamaindex.ai`.

Then, make sure you have the latest LlamaIndex version installed.

```
pip uninstall llama-index  # run this if upgrading from v0.9.x or older
pip install -U llama-index --upgrade --no-cache-dir --force-reinstall
```

Lastly, install the package:

`pip install llama-parse`

Now you can run the following to parse your first PDF file:

```python
import nest_asyncio

nest_asyncio.apply()

from llama_parse import LlamaParse

parser = LlamaParse(
    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
    result_type="markdown",  # "markdown" and "text" are available
    verbose=True,
)

# sync
documents = parser.load_data("./my_file.pdf")

# sync batch
documents = parser.load_data(["./my_file1.pdf", "./my_file2.pdf"])

# async
documents = await parser.aload_data("./my_file.pdf")

# async batch
documents = await parser.aload_data(["./my_file1.pdf", "./my_file2.pdf"])
```

## Using with `SimpleDirectoryReader`

You can also integrate the parser as the default PDF loader in `SimpleDirectoryReader`:

```python
import nest_asyncio

nest_asyncio.apply()

from llama_parse import LlamaParse
from llama_index.core import SimpleDirectoryReader

parser = LlamaParse(
    api_key="llx-...",  # can also be set in your env as LLAMA_CLOUD_API_KEY
    result_type="markdown",  # "markdown" and "text" are available
    verbose=True,
)

file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(
    "./data", file_extractor=file_extractor
).load_data()
```

Full documentation for `SimpleDirectoryReader` can be found on the [LlamaIndex Documentation](https://docs.llamaindex.ai/en/stable/module_guides/loading/simpledirectoryreader.html).

## Examples

Several end-to-end indexing examples can be found in the examples folder

- [Getting Started](https://github.com/run-llama/llama_parse/blob/main/examples/demo_basic.ipynb)
- [Advanced RAG Example](https://github.com/run-llama/llama_parse/blob/main/examples/demo_advanced.ipynb)
- [Raw API Usage](https://github.com/run-llama/llama_parse/blob/main/examples/demo_api.ipynb)
- [JSON MODE](https://github.com/run-llama/llama_parse/blob/main/examples/demo_json.ipynb)

## Terms of Service

See the [Terms of Service Here](https://github.com/run-llama/llama_parse/blob/main/TOS.pdf).
llama-index-packs-cohere-citation-chat 0.5.1
llama-index packs cohere_citation_chat integration
Requires Python: <4.0,>=3.9
Keywords: chat, citation, cite, cohere, engine, index

# Cohere Citations Chat Engine Pack

Creates and runs a custom `VectorStoreIndexWithCitationsChat` -- which provides the chat engine with documents/citation mode.
See the documentation [here](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) and [here](https://docs.cohere.com/docs/retrieval-augmented-generation-rag).

## CLI Usage

You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:

```bash
llamaindex-cli download-llamapack CohereCitationChatEnginePack --download-dir ./cohere_citation_chat_pack
```

You can then inspect the files at `./cohere_citation_chat_pack` and use them as a template for your own project!

You can also directly install it if you don't want to look at/inspect the source code:

```bash
pip install llama-index-packs-cohere-citation-chat
```

## Code Usage

You can download the pack to the `./cohere_citation_chat_pack` directory:

```python
from llama_index.readers.web import SimpleWebPageReader
from llama_index.core.llama_pack import download_llama_pack

# download and install dependencies
CohereCitationChatEnginePack = download_llama_pack(
    "CohereCitationChatEnginePack", "./cohere_citation_chat_pack"
)
documents = SimpleWebPageReader().load_data(
    [
        "https://raw.githubusercontent.com/jerryjliu/llama_index/adb054429f642cc7bbfcb66d4c232e072325eeab/examples/paul_graham_essay/data/paul_graham_essay.txt"
    ]
)
cohere_citation_chat_pack = CohereCitationChatEnginePack(
    documents=documents, cohere_api_key="your-api-key"
)
chat_engine = cohere_citation_chat_pack.run()
response = chat_engine.chat("What can you tell me about LLMs?")

# print chat response
print(response)

# print documents
print(response.documents)

# print citations
print(response.citations)
```

See the [notebook on llama](https://github.com/run-llama/llama_index/blob/main/llama-index-packs/llama-index-packs-cohere-citation-chat/examples/cohere_citation_chat_example.ipynb) for a full example.
llama-index-vector-stores-hnswlib 0.4.1
llama-index vector_stores hnswlib integration
Requires Python: <4.0,>=3.9

# LlamaIndex Vector_Stores Integration: Hnswlib

A LlamaIndex vector store using Hnswlib, a header-only C++ HNSW implementation with python bindings.

## Usage

Pre-requisite:

```
pip install llama-index-vector-stores-hnswlib
pip install hnswlib
```

A minimal example:

```python
import hnswlib
from llama_index.vector_stores_hnswlib import HnswlibVectorStore

space = "ip"  # distance function
dim = 768  # embedding dimension
hnswlib_index = hnswlib.Index(space, dim)
hnswlib_index.init_index(max_elements=10)

hnsw_vector_store = HnswlibVectorStore(hnswlib_index=index)
```

## More examples and references

A detailed usage guede can be found [in this demo notebook](https://docs.llamaindex.ai/en/stable/examples/vector_stores/HnswlibIndexDemo.html) in the LlamaIndex docs.

Hnswlib documentation and implementation can be found [here](https://github.com/nmslib/hnswlib).
llama-index-packs-dense-x-retrieval 0.5.1
llama-index packs dense_x_retrieval integration
Requires Python: <4.0,>=3.9

# Dense-X-Retrieval Pack

This LlamaPack creates a query engine that uses a `RecursiveRetriever` in llama-index to fetch nodes based on propoistions extracted from each node.

This follows the idea from the paper [Dense X Retrieval: What Retrieval Granularity Should We Use?](https://arxiv.org/abs/2312.06648).

From the paper, a proposition is described as:

```
Propositions are defined as atomic expressions within text, each encapsulating a distinct factoid and presented in a concise, self-contained natural language format.
```

We use the provided OpenAI prompt from their paper to generate propositions, which are then embedded and used to retrieve their parent node chunks.

**NOTE:** While the paper uses a fine-tuned model to extract propositions, it is unreleased at the time of writing. Currently, this pack uses the LLM to extract propositions, which can be expensive for large amounts of data.

## CLI Usage

You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:

```bash
llamaindex-cli download-llamapack DenseXRetrievalPack --download-dir ./dense_pack
```

You can then inspect the files at `./dense_pack` and use them as a template for your own project!

## Code Usage

You can download the pack to a the `./dense_pack` directory:

```python
from llama_index.core import SimpleDirectoryReader
from llama_index.core.llama_pack import download_llama_pack

# download and install dependencies
DenseXRetrievalPack = download_llama_pack(
    "DenseXRetrievalPack", "./dense_pack"
)

documents = SimpleDirectoryReader("./data").load_data()

# uses the LLM to extract propositions from every document/node!
dense_pack = DenseXRetrievalPack(documents)

# for streaming
dense_pack = DenseXRetrievalPack(documents, streaming=True)
```

The `run()` function is a light wrapper around `query_engine.query()`.

```python
response = dense_pack.run("What can you tell me about LLMs?")

print(response)
```

for streaming:

The `run()` function is a light wrapper around `query_engine.query()`.

```python
stream_response = dense_pack.run("What can you tell me about LLMs?")

stream_response.print_response_stream()
```

See the [notebook on llama-hub](https://github.com/run-llama/llama-hub/blob/main/llama_hub/llama_packs/dense_x_retrieval/dense_x_retrieval.ipynb) for a full example.
llama-index-readers-zendesk 0.4.1
llama-index readers zendesk integration
Requires Python: <4.0,>=3.9
Keywords: help center, knowledge base, zendesk

# Zendesk Loader

```bash
pip install llama-index-readers-zendesk
```

This loader fetches the text from Zendesk help articles using the Zendesk API. It also uses the BeautifulSoup library to parse the HTML and extract the text from the articles.

## Usage

To use this loader, you need to pass in the subdomain of a Zendesk account. No authentication is required. You can also set the locale of articles as needed.

```python
from llama_index.readers.zendesk import ZendeskReader

loader = ZendeskReader(zendesk_subdomain="my_subdomain", locale="en-us")
documents = loader.load_data()
```

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).