semvec/pypi-packages huggingface dataset:
llama-index-readers-joplin 0.5.1
llama-index readers joplin integration
Requires Python: <4.0,>=3.9

# Joplin (Markdown) Loader

```bash
pip install llama-index-readers-joplin
```

> [Joplin](https://joplinapp.org/) is an open source note-taking app. Capture your thoughts and securely access them from any device.

This readme covers how to load documents from a `Joplin` database.

`Joplin` has a [REST API](https://joplinapp.org/api/references/rest_api/) for accessing its local database. This reader uses the API to retrieve all notes in the database and their metadata. This requires an access token that can be obtained from the app by following these steps:

1. Open the `Joplin` app. The app must stay open while the documents are being loaded.
2. Go to settings / options and select "Web Clipper".
3. Make sure that the Web Clipper service is enabled.
4. Under "Advanced Options", copy the authorization token.

You may either initialize the reader directly with the access token, or store it in the environment variable JOPLIN_ACCESS_TOKEN.

An alternative to this approach is to export the `Joplin`'s note database to Markdown files (optionally, with Front Matter metadata) and use a Markdown reader, such as ObsidianReader, to load them.

## Usage

Here's an example usage of the JoplinReader.

```python
import os

from llama_index.readers.joplin import JoplinReader

documents = JoplinReader(
    access_token="<access_token>"
).load_data()  # Returns list of documents
```
llama-index-vector-stores-alibabacloud-opensearch 0.4.1
llama-index vector_stores Alibaba Cloud OpenSearch integration
Requires Python: <4.0,>=3.9

# LlamaIndex Vector_Stores Integration: Alibaba Cloud OpenSearch

Please refer to the [notebook](../../../docs/docs/examples/vector_stores/AlibabaCloudOpenSearchIndexDemo.ipynb) for usage of AlibabaCloud OpenSearch as vector store in LlamaIndex.

## Example Usage

```sh
pip install llama-index
pip install llama-index-vector-stores-alibabacloud-opensearch
```

```python
# Connect to existing instance
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.alibabacloud_opensearch import (
    AlibabaCloudOpenSearchStore,
    AlibabaCloudOpenSearchConfig,
)

config = AlibabaCloudOpenSearchConfig(
    endpoint="***",
    instance_id="***",
    username="your_username",
    password="your_password",
    table_name="llama",
)

vector_store = AlibabaCloudOpenSearchStore(config)

# Create index from existing stored vectors
index = VectorStoreIndex.from_vector_store(vector_store)
query_engine = index.as_query_engine()
response = query_engine.query(
    "What did the author study prior to working on AI?"
)
print(response)
```
llama-index-readers-reddit 0.4.1
llama-index readers reddit integration
Requires Python: <4.0,>=3.9
Keywords: comments, reddit, search, subreddit

# Reddit Reader

```bash
pip install llama-index-readers-reddit
```

For any subreddit(s) you're interested in, search for relevant posts using keyword(s) and load the resulting text in the post and and top-level comments into LLMs/ LangChains.

## Get your Reddit credentials ready

1. Visit Reddit App Preferences (https://www.reddit.com/prefs/apps) or [https://old.reddit.com/prefs/apps/](https://old.reddit.com/prefs/apps/)
2. Scroll to the bottom and click "create another app..."
3. Fill out the name, description, and redirect url for your app, then click "create app"
4. Now you should be able to see the personal use script, secret, and name of your app. Store those as environment variables REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET, and REDDIT_USER_AGENT respectively.
5. Additionally store the environment variables REDDIT_USERNAME and REDDIT_PASSWORD, which correspond to the credentials for your Reddit account.

## Usage

### LlamaIndex

```python
from llama_index.core import VectorStoreIndex, download_loader

from llama_index.readers.reddit import RedditReader

subreddits = ["MachineLearning"]
search_keys = ["PyTorch", "deploy"]
post_limit = 10

loader = RedditReader()
documents = loader.load_data(
    subreddits=subreddits, search_keys=search_keys, post_limit=post_limit
)
index = VectorStoreIndex.from_documents(documents)

index.query("What are the pain points of PyTorch users?")
```

### LangChain

```python
from llama_index.core import VectorStoreIndex, download_loader

from langchain.agents import initialize_agent, Tool
from langchain.llms import OpenAI
from langchain.chains.conversation.memory import ConversationBufferMemory

from llama_index.readers.reddit import RedditReader

subreddits = ["MachineLearning"]
search_keys = ["PyTorch", "deploy"]
post_limit = 10

loader = RedditReader()
documents = loader.load_data(
    subreddits=subreddits, search_keys=search_keys, post_limit=post_limit
)
index = VectorStoreIndex.from_documents(documents)

tools = [
    Tool(
        name="Reddit Index",
        func=lambda q: index.query(q),
        description=f"Useful when you want to read relevant posts and top-level comments in subreddits.",
    ),
]
llm = OpenAI(temperature=0)
memory = ConversationBufferMemory(memory_key="chat_history")
agent_chain = initialize_agent(
    tools, llm, agent="zero-shot-react-description", memory=memory
)

output = agent_chain.run(input="What are the pain points of PyTorch users?")
print(output)
```

This loader is designed to be used as a way to load data into [GPT Index](https://github.com/run-llama/llama_index/).
llama-index-tools-neo4j 0.4.1
llama-index tools neo4j integration
Requires Python: <4.0,>=3.9
Keywords: cypher, graph, neo4j

# Neo4j Schema Query Builder

```bash
pip install llama-index-tools-neo4j
```

The `Neo4jQueryToolSpec` class provides a way to query a Neo4j graph database based on a provided schema definition. The class uses a language model to generate Cypher queries from user questions and has the capability to recover from Cypher syntax errors through a self-healing mechanism.

## Table of Contents

- [Usage](#usage)
  - [Initialization](#initialization)
  - [Running a Query](#running-a-query)
- [Features](#features)

## Usage

### Initialization

Initialize the `Neo4jQueryToolSpec` class with:

```python
from llama_index.tools.neo4j import Neo4jQueryToolSpec
from llama_index.llms.openai import OpenAI
from llama_index.core.agent.workflow import FunctionAgent

gds_db = Neo4jQueryToolSpec(
    url="neo4j-url",
    user="neo4j-user",
    password="neo4j=password",
    llm=llm,
    database="neo4j",
)

tools = gds_db.to_tool_list()
agent = FunctionAgent(
    tools=tools,
    llm=OpenAI(model="gpt-4.1"),
)
```

Where:

- `url`: Connection string for the Neo4j database.
- `user`: Username for the Neo4j database.
- `password`: Password for the Neo4j database.
- `llm`: A language model for generating Cypher queries (any type of LLM).
- `database`: The database name.

### Running a Query

To use the agent:

```python
# use agent
resp = await agent.run("Where is JFK airport is located?")
```

```
Generated Cypher:

MATCH (p:Port {port_code: 'JFK'})
RETURN p.location_name_wo_diacritics AS Location

Final answer:
'The port code JFK is located in New York, United States.'
```

## Features

- **Schema-Based Querying**: The class extracts the Neo4j database schema to guide the Cypher query generation.
- **Self-Healing**: On a Cypher syntax error, the class corrects itself to produce a valid query.
- **Language Model Integration**: Uses a language model for natural and accurate Cypher query generation.
llama-index-readers-s3 0.5.1
llama-index readers s3 integration
Requires Python: <4.0,>=3.9
Keywords: amazon web services, aws s3, bucket

# S3 File or Directory Loader

This loader parses any file stored on S3, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing `S3Reader`, you may pass in your [AWS Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). If none are found, the loader assumes they are stored in `~/.aws/credentials`.

All files are parsed with `SimpleDirectoryReader`. Hence, you may also specify a custom `file_extractor`, relying on any of the loaders in this library (or your own)!

## Installation

```bash
pip install llama-index-readers-s3
```

## Usage

To use this loader, you need to pass in the name of your S3 Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like `subdirectory/input.txt`.

Otherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. AWS Access Key credentials may either be passed in during initialization or stored locally (see above).

```python
loader = S3Reader(
    bucket="scrabble-dictionary",
    key="dictionary.txt",
    aws_access_id="[ACCESS_KEY_ID]",
    aws_access_secret="[ACCESS_KEY_SECRET]",
)
documents = loader.load_data()
```

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
llama-index-readers-agent-search 0.4.1
llama-index readers agent_search integration
Requires Python: <4.0,>=3.9

# AgentSearch Loader

```bash
pip install llama-index-readers-agent-search
```

This framework facilitates seamless integration with the AgentSearch dataset or hosted search APIs (e.g. Search Engines) and with RAG-specialized LLM's (e.g. Search Agents).

During query-time, the user passes in the query string, search provider (`bing`, `agent-search`), and RAG provider model (`SciPhi/Sensei-7B-V1`).

To learn more, please refer to the documentation [here](https://agent-search.readthedocs.io/en/latest/).

## Usage

Here's an example usage of the AgentSearchReader.

```python
# Optionally set the API key in the env
# import os
# os.environ["SCIPHI_API_KEY"] = "..."

from llama_index.readers.agent_search import AgentSearchReader

reader = AgentSearch()

document = reader.load_data(
    query="latest news",
)[0]
# text = "The latest news encompasses ... and its consequences [2]."
# metadata = {'related_queries': ['Details on the...', ...], 'search_results' : [...]}
```

This loader is designed to be used as a way to load data into [LlamaIndex](https://github.com/run-llama/llama_index/).
https://github.com/run-llama/llama_index/
llama-index-packs-longrag 0.5.1
llama-index packs longrag integration
Requires Python: <4.0,>=3.9

# LlamaIndex Packs Integration: LongRAG

This LlamaPack implements LongRAG based on [this paper](https://arxiv.org/pdf/2406.15319).

LongRAG retrieves large tokens at a time, with each retrieval unit being ~6k tokens long, consisting of entire documents or groups of documents. This contrasts the short retrieval units (100 word passages) of traditional RAG. LongRAG is advantageous because results can be achieved using only the top 4-8 retrieval units, and long-context LLMs can better understand the context of the documents because long retrieval units preserve their semantic integrity.

## Installation

```
# installation
pip install llama-index-packs-longrag

# source code
llamaindex-cli download-llamapack LongRAGPack --download-dir ./longrag_pack
```

## Code Usage

```py
from llama_index.packs.longrag import LongRAGPack
from llama_index.llms.openai import OpenAI
from llama_index.core import Settings

Settings.llm = OpenAI("gpt-4o")

pack = LongRAGPack(data_dir="./data")

query_str = "How can Pittsburgh become a startup hub, and what are the two types of moderates?"
res = pack.run(query_str)
print(str(res))
```