semvec/pypi-packages huggingface dataset:
nlweb-retrieval 0.5.6
Retrieval bundle for NLWeb - includes vector database clients for multiple providers
License: MIT
Requires Python: >=3.9

# nlweb-retrieval

Bundle package containing all retrieval providers for NLWeb.

## Included Providers

This package includes clients for:

- **Azure AI Search** - Azure's vector search service with managed identity support
- **Elasticsearch** - Popular open-source search and analytics engine
- **Milvus** - Open-source vector database
- **Qdrant** - Vector similarity search engine
- **PostgreSQL** (pgvector) - PostgreSQL with vector extension
- **OpenSearch** - Open-source search and analytics suite
- **Snowflake Cortex Search** - Snowflake's vector search
- **Shopify MCP** - Shopify Model Context Protocol
- **Cloudflare AutoRAG** - Cloudflare's RAG service
- **Bing Search** - Microsoft Bing web search

## Installation

```bash
pip install nlweb-core nlweb-retrieval
```

## Configuration Example

Create `config.yaml`:

```yaml
retrieval:
  provider: elasticsearch
  import_path: nlweb_retrieval.elasticsearch_client
  class_name: ElasticsearchClient
  api_endpoint_env: ELASTICSEARCH_URL
  index_name: my_index
  use_knn: true
```

### Azure AI Search Example

```yaml
retrieval:
  provider: azure_ai_search
  import_path: nlweb_retrieval.azure_search_client
  class_name: AzureSearchClient
  api_endpoint_env: AZURE_SEARCH_ENDPOINT
  auth_method: azure_ad  # or api_key
  index_name: my-search-index
```

### Qdrant Example

```yaml
retrieval:
  provider: qdrant
  import_path: nlweb_retrieval.qdrant
  class_name: QdrantVectorClient
  database_path: ./data/qdrant_db  # for local storage
  # OR for remote:
  # api_endpoint_env: QDRANT_URL
  # api_key_env: QDRANT_API_KEY
```

## Usage

```python
import nlweb_core

# Initialize with config
nlweb_core.init(config_path="./config.yaml")

# Use retrieval
from nlweb_core import retriever

# Search
results = await retriever.search(
    query="example query",
    site="example.com",
    num_results=10
)
```

## Provider Import Paths

Use these in your config file:

| Provider | import_path | class_name |
|----------|-------------|------------|
| Azure AI Search | `nlweb_retrieval.azure_search_client` | `AzureSearchClient` |
| Elasticsearch | `nlweb_retrieval.elasticsearch_client` | `ElasticsearchClient` |
| Milvus | `nlweb_retrieval.milvus_client` | `MilvusVectorClient` |
| Qdrant | `nlweb_retrieval.qdrant` | `QdrantVectorClient` |
| PostgreSQL | `nlweb_retrieval.postgres_client` | `PgVectorClient` |
| OpenSearch | `nlweb_retrieval.opensearch_client` | `OpenSearchClient` |
| Snowflake | `nlweb_retrieval.snowflake_client` | `SnowflakeCortexSearchClient` |
| Shopify MCP | `nlweb_retrieval.shopify_mcp` | `ShopifyMCPClient` |
| Cloudflare | `nlweb_retrieval.cf_autorag_client` | `CloudflareAutoRAGClient` |
| Bing Search | `nlweb_retrieval.bing_search_client` | `BingSearchClient` |

## License

MIT License - Copyright (c) 2025 Microsoft Corporation
nlweb-qdrant-vectordb 0.5.5
Qdrant provider for NLWeb - third-party vector database provider
License: MIT
Requires Python: >=3.9

# nlweb-qdrant-vectordb

Qdrant provider for NLWeb.

## Overview

This package provides Qdrant vector database support for NLWeb, demonstrating how to create third-party provider packages.

## Installation

```bash
pip install nlweb-core nlweb-qdrant-vectordb
```

For LLM and embedding, you'll also need a model provider:
```bash
pip install nlweb-azure-models
```

Or use the bundle packages:
```bash
pip install nlweb-core nlweb-retrieval nlweb-models
```

## Configuration

Create `config.yaml`:

```yaml
retrieval:
  provider: qdrant
  import_path: nlweb_qdrant_vectordb.qdrant_client
  class_name: QdrantClient
  api_endpoint_env: QDRANT_URL  # Optional for remote Qdrant
  api_key_env: QDRANT_API_KEY  # Optional for remote Qdrant
  database_path_env: QDRANT_PATH  # Optional for local Qdrant
  index_name: my-collection
```

### Authentication

For remote Qdrant:
```bash
export QDRANT_URL=https://your-cluster.qdrant.tech
export QDRANT_API_KEY=your_api_key_here
```

For local Qdrant:
```bash
export QDRANT_PATH=./data/qdrant
```

## Usage

```python
import nlweb_core

# Initialize
nlweb_core.init(config_path="./config.yaml")

# Search
from nlweb_core import retriever

results = await retriever.search(
    query="example query",
    site="example.com",
    num_results=10
)
```

## Features

- Vector similarity search with Qdrant
- Support for both remote and local Qdrant instances
- HNSW-based efficient similarity search
- Configurable collection names
- API key authentication for remote instances
- Local file-based storage option
- Compatible with NLWeb Protocol v0.5

## Creating Your Own Provider Package

Use this package as a template:

1. **Create package structure**:
   ```
   nlweb-yourprovider/
   ├── pyproject.toml
   ├── README.md
   └── nlweb_yourprovider/
       ├── __init__.py
       └── your_client.py
   ```

2. **Implement VectorDBClientInterface**:
   ```python
   from nlweb_core.retriever import VectorDBClientInterface

   class YourClient(VectorDBClientInterface):
       async def search(self, query, site, num_results, **kwargs):
           # Your implementation
           pass
   ```

3. **Declare dependencies** in `pyproject.toml`:
   ```toml
   dependencies = [
       "nlweb-core>=0.5.0",
       "your-provider-sdk>=1.0.0",
   ]
   ```

4. **Publish to PyPI**:
   ```bash
   python -m build
   twine upload dist/*
   ```

## License

MIT License - Copyright (c) 2025 Microsoft Corporation
nlweb-elastic-vectordb 0.5.5
Elasticsearch provider for NLWeb - third-party vector database provider
License: MIT
Requires Python: >=3.9

# nlweb-elastic-vectordb

Elasticsearch provider for NLWeb.

## Overview

This package provides Elasticsearch vector database support for NLWeb, demonstrating how to create third-party provider packages.

## Installation

```bash
pip install nlweb-core nlweb-elastic-vectordb
```

For LLM and embedding, you'll also need a model provider:
```bash
pip install nlweb-azure-models
```

Or use the bundle packages:
```bash
pip install nlweb-core nlweb-retrieval nlweb-models
```

## Configuration

Create `config.yaml`:

```yaml
retrieval:
  provider: elasticsearch
  import_path: nlweb_elastic_vectordb.elasticsearch_client
  class_name: ElasticsearchClient
  api_endpoint_env: ELASTICSEARCH_ENDPOINT
  api_key_env: ELASTICSEARCH_API_KEY
  index_name: my-index
```

### Authentication

Set environment variables:
```bash
export ELASTICSEARCH_ENDPOINT=https://your-cluster.elasticsearch.com
export ELASTICSEARCH_API_KEY=your_api_key_here
```

## Usage

```python
import nlweb_core

# Initialize
nlweb_core.init(config_path="./config.yaml")

# Search
from nlweb_core import retriever

results = await retriever.search(
    query="example query",
    site="example.com",
    num_results=10
)
```

## Features

- Vector similarity search with Elasticsearch
- KNN search with filtering
- Configurable index names
- API key authentication
- Compatible with NLWeb Protocol v0.5

## Creating Your Own Provider Package

Use this package as a template:

1. **Create package structure**:
   ```
   nlweb-yourprovider/
   ├── pyproject.toml
   ├── README.md
   └── nlweb_yourprovider/
       ├── __init__.py
       └── your_client.py
   ```

2. **Implement VectorDBClientInterface**:
   ```python
   from nlweb_core.retriever import VectorDBClientInterface

   class YourClient(VectorDBClientInterface):
       async def search(self, query, site, num_results, **kwargs):
           # Your implementation
           pass
   ```

3. **Declare dependencies** in `pyproject.toml`:
   ```toml
   dependencies = [
       "nlweb-core>=0.5.0",
       "your-provider-sdk>=1.0.0",
   ]
   ```

4. **Publish to PyPI**:
   ```bash
   python -m build
   twine upload dist/*
   ```

## License

MIT License - Copyright (c) 2025 Microsoft Corporation
fastapi-mongo-logger-mateoramos 1.0.7
FastAPI MongoDB logging package for endpoints and general logging
License: MIT
Requires Python: <4.0,>=3.7
Keywords: fastapi, mongodb, logging, middleware, async, jwt

# FastAPI MongoDB Logger

A comprehensive logging package for FastAPI applications that stores endpoint logs and custom events in MongoDB.

## Installation

```bash
pip install fastapi-mongo-logger-mateoramos
```

## Quick Start

### 1. Basic Setup

```python
from fastapi import FastAPI
from fastapi_mongo_logger import MongoLogger, LoggingMiddleware

app = FastAPI()

# Initialize logger
logger = MongoLogger(
    mongo_url="mongodb://localhost:27017",
    database_name="my_app_logs",
    collection_name="api_logs"
)

# Add middleware for automatic endpoint logging
app.add_middleware(LoggingMiddleware, logger=logger)

@app.get("/")
async def root():
    return {"message": "Hello World"}
```

### 2. Manual Logging with Decorators

```python
from fastapi_mongo_logger import log_endpoint, log_function

@log_endpoint(logger, user_type="admin")
async def admin_function():
    # Your code here
    pass

@log_function(logger, event_type="data_processing")
def process_data():
    # Your code here
    pass
```

### 3. Custom Logging

```python
# Log custom events anywhere in your code
await logger.log_custom("user_action", {
    "user_id": "123",
    "action": "login",
    "ip_address": "192.168.1.1"
})
```

## Features

- **Automatic endpoint logging** via middleware
- **Manual logging** with decorators
- **Custom event logging** for any part of your application
- **Comprehensive data capture**: request/response bodies, headers, timing, errors
- **Async/sync function support**
- **Flexible data storage** in MongoDB

## Configuration Options

- `log_request_body`: Enable/disable request body logging (default: True)
- `log_response_body`: Enable/disable response body logging (default: False)
- Custom fields can be added to any log entry
pgmate 0.1.0
PostgreSQL All-in-One Toolkit (Queue + Cron + KV Store + PubSub)
License: MIT
Requires Python: >=3.8

# PgMate

**PgMate** (formerly PgKit) is a lightweight PostgreSQL All-in-One Toolkit that provides:
- **Queue**: Simple job queue backed by PostgreSQL.
- **Cron**: Scheduled tasks using `pg_cron`.
- **KV Store**: Key-Value store with TTL support.
- **PubSub**: Real-time event notifications using `LISTEN/NOTIFY`.

## Requirements

- Python 3.8+
- PostgreSQL database
- `pg_cron` extension enabled in PostgreSQL (required for Cron and KV cleanup features)

## Installation

```bash
pip install pgmate
```

## Usage

### Initialization

```python
from pgmate import PgMate

# Initialize with PostgreSQL DSN
# Automatically creates necessary tables (_pgmate_jobs, _pgmate_kv) if they don't exist
mate = PgMate(dsn="postgres://user:password@localhost:5432/dbname")
```

### Job Queue

**Producer:**

```python
# Enqueue a job
job_id = mate.enqueue(
    queue_name="my_queue",
    payload={"task": "send_email", "email": "user@example.com"},
    delay_seconds=0
)
print(f"Job enqueued with ID: {job_id}")
```

**Consumer:**

```python
def process_job(payload):
    print(f"Processing: {payload}")
    # Raise exception to retry the job

# Start a worker that listens for new jobs
# This is a blocking call
mate.listen_and_consume("my_queue", process_job)
```

### Key-Value Store

```python
# Set a value with TTL (optional)
mate.kv_set("session:123", {"user_id": 42}, ttl_seconds=3600)

# Get a value
value = mate.kv_get("session:123")
print(value)  # {'user_id': 42}

# Delete a value
mate.kv_delete("session:123")

# Enable auto-cleanup for expired keys (requires pg_cron)
mate.enable_kv_cleanup_job()
```

### Cron Scheduler

Schedule recurring jobs using cron expressions.

```python
# Schedule a job to run every minute
mate.schedule_cron(
    cron_expression="* * * * *",
    job_name="daily_report",
    queue_name="reports",
    payload={"type": "daily"}
)
```

## Architecture

- **Tables**: `_pgmate_jobs`, `_pgmate_kv`
- **Notifications**: Uses `LISTEN/NOTIFY` on channel `_pgmate_event_new_job` for low-latency job processing.
- **Concurrency**: Safe for multiple workers (uses `FOR UPDATE SKIP LOCKED`).