semvec/pypi-packages huggingface dataset:
llama-index-tools-brave-search 0.4.1
llama-index tools brave_search integration
Requires Python: <4.0,>=3.9

# LlamaIndex Tools Integration: Brave_Search

This tool enables agents to search and retrieve results from the Brave search engine.

You will need to set up an Brave account to get an search api key. Please check more here: https://brave.com/search/api

## Usage

This tool has a more extensive example usage documented in a Jupyter notebook [here](./examples/brave_search.ipynb)

Here's an example usage of the BraveSearchToolSpec.

```python
from llama_index.tools.brave_search import BraveSearchToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

tool_spec = BraveSearchToolSpec(api_key="your-key")

agent = FunctionAgent(
    tools=tool_spec.to_tool_list(),
    llm=OpenAI(model="gpt-4.1"),
)

answer = await agent.run("what's the latest news about superconductors")
answer = await agent.run("what does lk-99 look like")
answer = await agent.run("is there any videos of it levitating")
```
llama-index-llms-clarifai 0.4.1
llama-index llms clarifai integration
Requires Python: <4.0,>=3.9

# LlamaIndex Llms Integration: Clarifai

### Installation

```bash
%pip install llama-index-llms-clarifai
!pip install llama-index
!pip install clarifai
```

### Basic Usage

```py
# Set Clarifai PAT as an environment variable
import os

os.environ["CLARIFAI_PAT"] = "<YOUR CLARIFAI PAT>"

# Import Clarifai package
from llama_index.llms.clarifai import Clarifai

# Example parameters
params = dict(
    user_id="clarifai",
    app_id="ml",
    model_name="llama2-7b-alternative-4k",
    model_url="https://clarifai.com/clarifai/ml/models/llama2-7b-alternative-4k",
)

# Initialize the LLM
# Method:1 using model_url parameter
llm_model = Clarifai(model_url=params["model_url"])

# Method:2 using model_name, app_id & user_id parameters
llm_model = Clarifai(
    model_name=params["model_name"],
    app_id=params["app_id"],
    user_id=params["user_id"],
)

# Call complete function
llm_response = llm_model.complete(
    prompt="write a 10 line rhyming poem about science"
)
print(llm_response)

# Output
# Science is fun, it's true!
# From atoms to galaxies, it's all new!
# With experiments and tests, we learn so fast,
# And discoveries come from the past.
# It helps us understand the world around,
# And makes our lives more profound.
# So let's embrace this wondrous art,
# And see where it takes us in the start!
```

### Call chat function

```py
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="user", content="write about climate change in 50 lines")
]
response = llm_model.chat(messages)
print(response)

# Output
# user: or less.
# Climate change is a serious threat to our planet and its inhabitants.
# Rising temperatures are causing extreme weather events, such as hurricanes,
# droughts, and wildfires. Sea levels are rising, threatening coastal
# communities and ecosystems. The melting of polar ice caps is disrupting
# global navigation and commerce. Climate change is also exacerbating air
# pollution, which can lead to respiratory problems and other health issues.
# It's essential that we take action now to reduce greenhouse gas emissions
# and transition to renewable energy sources to mitigate the worst effects
# of climate change.
```

### Using Inference parameters

```py
# Alternatively, you can call models with inference parameters.

# Here is an inference parameter example for GPT model.
inference_params = dict(temperature=str(0.3), max_tokens=20)

llm_response = llm_model.complete(
    prompt="What is nuclear fission and fusion?",
    inference_params=inference_params,
)

messages = [ChatMessage(role="user", content="Explain about the big bang")]
response = llm_model.chat(messages, inference_params=inference_params)

print(response)
```

### LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/clarifai/
llama-index-utils-qianfan 0.4.1
llama-index utils baidu qianfan integration
Requires Python: <4.0,>=3.9

# LlamaIndex Utils Integration: Baidu Qianfan

Client-side underlying components of Baidu Intelligent Cloud's Qianfan LLM Platform. Can be used to support access to model services of types such as chat, completion, and embedding.
llama-index-llms-litellm 0.6.3
llama-index llms litellm integration
Requires Python: <4.0,>=3.9

# LlamaIndex Llms Integration: Litellm

## Installation

1. Install the required Python packages:

   ```bash
   %pip install llama-index-llms-litellm
   !pip install llama-index
   ```

## Usage

### Import Required Libraries

```python
import os
from llama_index.llms.litellm import LiteLLM
from llama_index.core.llms import ChatMessage
```

### Set Up Environment Variables

Set your API keys as environment variables:

```python
os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["COHERE_API_KEY"] = "your-api-key"
```

### Example: OpenAI Call

To interact with the OpenAI model:

```python
message = ChatMessage(role="user", content="Hey! how's it going?")
llm = LiteLLM("gpt-3.5-turbo")
chat_response = llm.chat([message])
print(chat_response)
```

### Example: Cohere Call

To interact with the Cohere model:

```python
llm = LiteLLM("command-nightly")
chat_response = llm.chat([message])
print(chat_response)
```

### Example: Chat with System Message

To have a chat with a system role:

```python
messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]
resp = LiteLLM("gpt-3.5-turbo").chat(messages)
print(resp)
```

### Streaming Responses

To use the streaming feature with `stream_complete`:

```python
llm = LiteLLM("gpt-3.5-turbo")
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")
```

### Streaming Chat Example

To stream chat messages:

```python
llm = LiteLLM("gpt-3.5-turbo")
resp = llm.stream_chat(messages)
for r in resp:
    print(r.delta, end="")
```

### Asynchronous Example

For asynchronous calls, use:

```python
llm = LiteLLM("gpt-3.5-turbo")
resp = await llm.acomplete("Paul Graham is ")
print(resp)
```

### LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/litellm/
llama-index-readers-notion 0.4.1
llama-index readers notion integration
Requires Python: <4.0,>=3.9

# LlamaIndex Readers Integration: Notion

## Overview

Notion Page Reader enables loading data from Notion pages. It constructs queries to retrieve pages based on page IDs or from a specified Notion database.

### Installation

You can install Notion Reader via pip:

```bash
pip install llama-index-readers-notion
```

### Usage

```python
from llama_index.readers.notion import NotionPageReader

# Initialize NotionPageReader
reader = NotionPageReader(integration_token="<Integration Token>")

# Load data from Notion
documents = reader.load_data(
    page_ids=["<Page ID 1>", "<Page ID 2>"],  # List of page IDs to load
    database_id="<Database ID>",  # Database ID from which to load page IDs
)
```

Implementation for Notion reader can be found [here](https://docs.llamaindex.ai/en/stable/examples/data_connectors/NotionDemo/)

This loader is designed to be used as a way to load data into
[LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently
used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent.
llama-index-readers-airbyte-stripe 0.4.1
llama-index readers airbyte_stripe integration
Requires Python: <4.0,>=3.9

# Airbyte Stripe Loader

```bash
pip install llama-index-readers-airbyte-stripe
```

The Airbyte Stripe Loader allows you to access different Stripe objects.

## Usage

Here's an example usage of the AirbyteStripeReader.

```python
from llama_index.readers.airbyte_stripe import AirbyteStripeReader

stripe_config = {
    # ...
}
reader = AirbyteStripeReader(config=stripe_config)
documents = reader.load_data(stream_name="invoices")
```

## Configuration

Check out the [Airbyte documentation page](https://docs.airbyte.com/integrations/sources/stripe/) for details about how to configure the reader.
The JSON schema the config object should adhere to can be found on Github: [https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-stripe/source_stripe/spec.yaml](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-stripe/source_stripe/spec.yaml).

The general shape looks like this:

```python
{
    "client_secret": "<secret key>",
    "account_id": "<account id>",
    "start_date": "<date from which to start retrieving records from in ISO format, e.g. 2020-10-20T00:00:00Z>",
}
```

By default all fields are stored as metadata in the documents and the text is set to the JSON representation of all the fields. Construct the text of the document by passing a `record_handler` to the reader:

```python
def handle_record(record, id):
    return Document(
        doc_id=id, text=record.data["title"], extra_info=record.data
    )


reader = AirbyteStripeReader(
    config=stripe_config, record_handler=handle_record
)
```

## Lazy loads

The `reader.load_data` endpoint will collect all documents and return them as a list. If there are a large number of documents, this can cause issues. By using `reader.lazy_load_data` instead, an iterator is returned which can be consumed document by document without the need to keep all documents in memory.

## Incremental loads

This loader supports loading data incrementally (only returning documents that weren't loaded last time or got updated in the meantime):

```python
reader = AirbyteStripeReader(config={...})
documents = reader.load_data(stream_name="invoices")
current_state = reader.last_state  # can be pickled away or stored otherwise

updated_documents = reader.load_data(
    stream_name="invoices", state=current_state
)  # only loads documents that were updated since last time
```

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
llama-index-llms-opea 0.2.1
llama-index llms opea integration
Requires Python: <4.0,>=3.9

# LlamaIndex Llms Integration: OPEA LLM

OPEA (Open Platform for Enterprise AI) is a platform for building, deploying, and scaling AI applications. As part of this platform, many core gen-ai components are available for deployment as microservices, including LLMs.

Visit [https://opea.dev](https://opea.dev) for more information, and their [GitHub](https://github.com/opea-project/GenAIComps) for the source code of the OPEA components.

## Installation

1. Install the required Python packages:

```bash
%pip install llama-index-llms-opea
```

## Usage

```python
from llama_index.core.llms import ChatMessage
from llama_index.llms.opea import OPEA

llm = OPEA(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    api_base="http://localhost:8080/v1",
    temperature=0.7,
    max_tokens=256,
    additional_kwargs={"top_p": 0.95},
)

# Complete a prompt
response = llm.complete("What is the capital of France?")
print(response)

# Stream a chat response
response = llm.stream_chat(
    [ChatMessage(role="user", content="What is the capital of France?")]
)
for chunk in response:
    print(chunk.delta, end="", flush=True)
```

All available methods include:

- `complete()`
- `stream_complete()`
- `chat()`
- `stream_chat()`

as well as async versions of the methods with the `a` prefix.
llama-index-readers-rayyan 0.4.1
llama-index readers rayyan integration
Requires Python: <4.0,>=3.9
Keywords: rayyan, systematic review

# Rayyan Loader

```bash
pip install llama-index-readers-rayyan
```

This loader fetches review articles from [Rayyan](https://www.rayyan.ai/)
using the [Rayyan SDK](https://github.com/rayyansys/rayyan-python-sdk). All articles
for a given review are fetched by default unless a filter is specified.

## Usage

To use this loader, you need to specify the path to the Rayyan credentials file
and optionally the API server URL if different from the default. More details
about these parameters can be found in the official Rayyan SDK repository.

```python
from llama_index.readers.rayyan import RayyanReader

loader = RayyanReader(credentials_path="path/to/rayyan-creds.json")
```

Once the loader is initialized, you can load data from Rayyan, either all or filtered:

```python
# Load all documents for a review with ID 123456
documents = loader.load_data(review_id=123456)

# Load only those that contain the word "outcome"
documents = loader.load_data(
    review_id=123456, filters={"search[value]": "outcome"}
)
```

The Rayyan SDK has more information about the available filters.

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
llama-index-tools-shopify 0.4.1
llama-index tools shopify integration
Requires Python: <4.0,>=3.9

# Shopify Tool

This tool acts as a custom app for Shopify stores, allowing the Agent to execute GraphQL queries to gather information or perform mutations against the Shopify store.

## Usage

This tool has more extensive example usage documented in a Jupyter notebook [here](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/tools/llama-index-tools-shopify/examples/shopify.ipynb)

In particular, the tool is very effective when combined with a method of retrieving data from the GraphQL schema definition.

```bash
pip install llama-index llama-index-readers-file llama-index-tools-shopify unstructured
```

```python
from llama_index.tools.shopify import ShopifyToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

from llama_index.readers.file import UnstructuredReader
from llama_index.core.tools.ondemand_loader_tool import OnDemandLoaderTool

documentation_tool = OnDemandLoaderTool.from_defaults(
    UnstructuredReader(),
    name="graphql_writer",
    description="""
        The GraphQL schema file is located at './data/shopify_graphql.txt', this is always the file argument.
        A tool for processing the Shopify GraphQL spec, and writing queries from the documentation.

        You should pass a query_str to this tool in the form of a request to write a GraphQL query.

        Examples:
            file: './data/shopify_graphql.txt', query_str='Write a graphql query to find unshipped orders'
            file: './data/shopify_graphql.txt', query_str='Write a graphql query to retrieve the stores products'
            file: './data/shopify_graphql.txt', query_str='What fields can you retrieve from the orders object'
        """,
)


shopify_tool = ShopifyToolSpec(
    "your-store.myshopify.com", "2023-04", "your-api-key"
)

agent = FunctionAgent(
    tools=[*shopify_tool.to_tool_list(), documentation_tool],
    llm=OpenAI(model="gpt-4.1"),
    system_prompt=f"""
    You are a specialized Agent with access to the Shopify Admin GraphQL API for this Users online store.
    Your job is to chat with store owners and help them run GraphQL queries, interpreting the results for the user

    You can use graphql_writer to query the schema and assist in writing queries.

    If the GraphQL you execute returns an error, either directly fix the query, or directly ask the graphql_writer questions about the schema instead of writing graphql queries.
    Then use that information to write the correct graphql query
    """,
)

print(await agent.run("What products are in my store?"))
```

`run_graphql_query`: Executes a GraphQL query against the Shopify store

This loader is designed to be used as a way to load data as a Tool in a Agent.
llama-index-packs-fuzzy-citation 0.4.1
llama-index packs fuzzy_citation integration
Requires Python: <4.0,>=3.9
Keywords: citation, cite, engine, fuzzy, index, query

# Fuzzy Citation Query Engine Pack

Creates and runs a `CustomQueryEngine` -- `FuzzCitationQueryEngine` -- which post-processes response objects to identify source sentences using fuzzy matching.

The identified sentences are available in the `response.metadata` dictionary, containing a mapping of `(response_sentence, source_chunk)` -> `{"start_char_idx": idx, "end_char_idx": idx, "node" node}`. The start/end idxs represent the character indexes in the node text that the source chunk comes from.

The fuzzy matching uses `fuzz.ratio()` to compare sentences. The default threshold score is 50.

## CLI Usage

You can download llamapacks directly using `llamaindex-cli`, which comes installed with the `llama-index` python package:

```bash
llamaindex-cli download-llamapack FuzzyCitationEnginePack --download-dir ./fuzzy_citation_pack
```

You can then inspect the files at `./fuzzy_citation_pack` and use them as a template for your own project!

## Code Usage

You can download the pack to a the `./fuzzy_citation_pack` directory:

```python
from llama_index.core import Document, VectorStoreIndex
from llama_index.core.llama_pack import download_llama_pack

# download and install dependencies
FuzzyCitationEnginePack = download_llama_pack(
    "FuzzyCitationEnginePack", "./fuzzy_citation_pack"
)

index = VectorStoreIndex.from_documents([Document.example()])
query_engine = index.as_query_engine()

fuzzy_engine = FuzzyCitationEnginePack(query_engine, threshold=50)
```

The `run()` function is a light wrapper around `query_engine.query()`. The response will have metadata attached to it indicating the fuzzy citations.

```python
response = fuzzy_engine.run("What can you tell me about LLMs?")

# print source sentences
print(response.metadata.keys())

# print full source sentence info
print(response.metadata)
```

See the [notebook on llama-hub](https://github.com/run-llama/llama-hub/blob/main/llama_hub/llama_packs/fuzzy_citation/fuzzy_citation_example.ipynb) for a full example.
llama-index-packs-searchain 0.2.1
llama-index packs searchain integration
Requires Python: <4.0,>=3.9
Keywords: llm, searchain

# LlamaIndex Packs Integration: Searchain

This LlamaPack implements a framework called SearChain, which implements the interaction between LLM and IR in the form of the global reasoning chain called Chain-of-Query (CoQ).

This follows the idea in the paper [Search-in-the-Chain: Towards Accurate, Credible and Traceable Large Language Models for Knowledge-intensive Tasks](https://arxiv.org/abs/2304.14732).

Making content generated by large language models (LLMs) such as ChatGPT accurate, trustworthy, and traceable is critical, especially for knowledge-intensive tasks. Introducing information retrieval (IR) to provide LLM with external knowledge is likely to solve this problem, however, where and how to introduce IR is a big challenge. The SearChain framework generates a global reasoning chain called a Chain of Query (CoQ) for LLM, where each node contains an IR-oriented query and the answer to the query. Second, IR verifies the answer of each node of CoQ, it corrects the answer that is not consistent with the retrieved information when IR gives high confidence, which improves the credibility. Third, LLM can mark its missing knowledge in CoQ and IR can provide this knowledge to LLM. These three operations improve the accuracy of LLM for complex knowledge-intensive tasks in terms of reasoning ability and knowledge. This Pack implements the above🤗!

You can see its use case in the examples folder.

This implementation is adapted from the author's implementation. You can find the official code repository [here](https://github.com/xsc1234/Search-in-the-Chain).

## Code Usage

First, you need to install SearChainpack using the following code,

```python
from llama_index.core.llama_pack import download_llama_pack

download_llama_pack("SearChainPack", "./searchain_pack")
```

Next you can load and initialize a searchain object,

```python
from searchain_pack.base import SearChainPack

searchain = SearChainPack(
    data_path="data",
    dprtokenizer_path="dpr_reader_multi",
    dprmodel_path="dpr_reader_multi",
    crossencoder_name_or_path="Quora_cross_encoder",
)
```

Relevant data can be found [here](https://www.kaggle.com/datasets/anastasiajia/searchain/data). You can run searchain using the following method,

```python
start_idx = 0
while not start_idx == -1:
    start_idx = execute(
        "/hotpotqa/hotpot_dev_fullwiki_v1_line.json", start_idx=start_idx
    )
```
llama-index-readers-apify 0.4.1
llama-index readers apify integration
Requires Python: <4.0,>=3.9
Keywords: apify, crawler, scraper, scraping

# Apify Loaders

```bash
pip install llama-index-readers-apify
```

## Apify Actor Loader

[Apify](https://apify.com/) is a cloud platform for web scraping and data extraction,
which provides an [ecosystem](https://apify.com/store) of more than a thousand
ready-made apps called _Actors_ for various scraping, crawling, and extraction use cases.

This loader runs a specific Actor and loads its results.

## Usage

In this example, we’ll use the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor,
which can deeply crawl websites such as documentation, knowledge bases, help centers,
or blogs, and extract text content from the web pages.
The extracted text then can be fed to a vector index or language model like GPT
in order to answer questions from it.

To use this loader, you need to have a (free) Apify account
and set your [Apify API token](https://console.apify.com/account/integrations) in the code.

```python
from llama_index.core import Document
from llama_index.readers.apify import ApifyActor

reader = ApifyActor("<My Apify API token>")

documents = reader.load_data(
    actor_id="apify/website-content-crawler",
    run_input={
        "startUrls": [{"url": "https://docs.llamaindex.ai/en/latest/"}]
    },
    dataset_mapping_function=lambda item: Document(
        text=item.get("text"),
        metadata={
            "url": item.get("url"),
        },
    ),
)
```

This loader is designed to be used as a way to load data into
[LlamaIndex](https://github.com/run-llama/llama_index/tree/main/llama_index) and/or subsequently
used as a Tool in a [LangChain](https://github.com/hwchase17/langchain) Agent.

## Apify Dataset Loader

[Apify](https://apify.com/) is a cloud platform for web scraping and data extraction,
which provides an [ecosystem](https://apify.com/store) of more than a thousand
ready-made apps called _Actors_ for various scraping, crawling, and extraction use cases.

This loader loads documents from an existing [Apify dataset](https://docs.apify.com/platform/storage/dataset).

## Usage

In this example, we’ll load a dataset generated by
the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor,
which can deeply crawl websites such as documentation, knowledge bases, help centers,
or blogs, and extract text content from the web pages.
The extracted text then can be fed to a vector index or language model like GPT
in order to answer questions from it.

To use this loader, you need to have a (free) Apify account
and set your [Apify API token](https://console.apify.com/account/integrations) in the code.

```python
from llama_index.core import Document
from llama_index.readers.apify import ApifyDataset

reader = ApifyDataset("<Your Apify API token>")
documents = reader.load_data(
    dataset_id="<Apify Dataset ID>",
    dataset_mapping_function=lambda item: Document(
        text=item.get("text"),
        metadata={
            "url": item.get("url"),
        },
    ),
)
```