semvec/pypi-packages huggingface dataset:
vision-agents-plugins-elevenlabs 0.3.6
ElevenLabs TTS and STT integration for Vision Agents
Requires Python: >=3.10
Keywords: AI, STT, TTS, agents, elevenlabs, speech-to-text, text-to-speech, voice agents

# ElevenLabs Text-to-Speech Plugin

A high-quality Text-to-Speech (TTS) plugin for GetStream that uses the ElevenLabs API.

## Installation

```bash
pip install getstream-plugins-elevenlabs
```

## Usage

```python
from getstream.plugins.elevenlabs import ElevenLabsTTS
from getstream.video.rtc.audio_track import AudioStreamTrack

# Initialize with API key from environment variable
tts = ElevenLabsTTS()

# Or specify API key directly
tts = ElevenLabsTTS(api_key="your_elevenlabs_api_key")

# Create an audio track to output speech
track = AudioStreamTrack(framerate=16000)
tts.set_output_track(track)

# Register event handlers
@tts.on("audio")
def on_audio(audio_data, user):
    print(f"Received audio chunk: {len(audio_data)} bytes")

# Send text to be converted to speech
await tts.send("Hello, this is a test of the ElevenLabs text-to-speech plugin.")
```

## Configuration Options

- `api_key`: ElevenLabs API key (default: reads from ELEVENLABS_API_KEY environment variable)
- `voice_id`: The voice ID to use for synthesis (default: "VR6AewLTigWG4xSOukaG")
- `model_id`: The model ID to use for synthesis (default: "eleven_multilingual_v2")

## Requirements

- Python 3.10+
- elevenlabs>=2.5.0
vision-agents-plugins-nvidia 0.3.6
NVIDIA VLM plugin for Vision Agents
Requires Python: >=3.10
Keywords: AI, VLM, agents, nvcf, nvidia, vision agents

# NVIDIA Plugin for Vision Agents

NVIDIA VLM integration for Vision Agents. Supports vision language models through NVIDIA's Chat Completions API with NVCF asset management.

## Features

- Video understanding: Automatically buffers and forwards video frames to NVIDIA VLM models
- Streaming responses: Supports streaming text responses with real-time chunk events
- Asset management: Automatically uploads frames as assets and cleans them up after use
- Configurable frame rate and buffer duration for optimal performance

## Installation

```bash
uv add vision-agents[nvidia]
```

## Configuration

Set your NVIDIA API key:

```bash
export NVIDIA_API_KEY=your_nvidia_api_key
```

## Usage

### Basic VLM Usage

```python
from vision_agents.plugins import nvidia

vlm = nvidia.VLM(
    model="nvidia/cosmos-reason2-8b",
    fps=1,
    frame_buffer_seconds=10,
)

# VLM automatically buffers video frames when used with an Agent
response = await vlm.simple_response("What do you see?")
print(response.text)
```

### With Custom Configuration

```python
from vision_agents.plugins import nvidia

vlm = nvidia.VLM(
    model="nvidia/cosmos-reason2-8b",
    api_key="your-api-key",
    fps=2,
    frame_buffer_seconds=15,
    frame_width=1280,
    frame_height=720,
    max_tokens=2048,
    temperature=0.3,
    top_p=0.8,
)
```

## Configuration

| Parameter | Description | Default | Type |
|-----------|-------------|---------|------|
| `model` | NVIDIA model ID | `"nvidia/cosmos-reason2-8b"` | str |
| `api_key` | NVIDIA API token (or use `NVIDIA_API_KEY` env var) | `None` | Optional[str] |
| `fps` | Frames per second to buffer | `1` | int |
| `frame_buffer_seconds` | Number of seconds to buffer | `10` | int |
| `frame_width` | Width of video frames to send | `800` | int |
| `frame_height` | Height of video frames to send | `600` | int |
| `max_tokens` | Maximum response tokens | `1024` | int |
| `temperature` | Temperature for sampling | `0.2` | float |
| `top_p` | Top-p sampling parameter | `0.7` | float |
| `frames_per_second` | Frames per second for video models | `8` | int |

## Dependencies

- `vision-agents`: Core Vision Agents framework
- `aiohttp>=3.9.0`: Async HTTP client for API requests
vision-agents-plugins-qwen 0.3.6
Qwen Omni plugin for vision agents
Requires Python: >=3.10

# Qwen Realtime Plugin for Vision Agents

Qwen3 Realtime LLM integration for Vision Agents framework with native audio output and built-in speech recognition using WebSocket-based realtime communication.

## Features

- **Native audio output**: No TTS service needed - audio comes directly from the model
- **Built-in STT**: Integrated speech-to-text using `gummy-realtime-v1` - no external STT service required
- **Server-side VAD**: Automatic turn detection with configurable silence thresholds
- **Video understanding**: Optional video frame support for multimodal interactions
- **Real-time streaming**: WebSocket-based bidirectional communication for low-latency responses
- **Interruption handling**: Automatic cancellation when user starts speaking

## Installation

```bash
uv add vision-agents[qwen]
```

## Usage

```python
from vision_agents.core import User, Agent
from vision_agents.plugins import getstream, qwen

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Qwen Assistant"),
    instructions="Be helpful and friendly",
    llm=qwen.Realtime(
        model="qwen3-omni-flash-realtime",
        voice="Cherry",
        fps=1,
    ),
    # No STT or TTS needed - Qwen Realtime provides both
)
```

## Configuration

| Parameter | Description | Default | Accepted Values |
|-----------|-------------|---------|----------------|
| `model` | Qwen Realtime model identifier | `"qwen3-omni-flash-realtime"` | Model name string |
| `api_key` | DashScope API key | `None` (from env) | String or `None` |
| `base_url` | WebSocket API base URL | `"wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime"` | URL string |
| `voice` | Voice for audio output | `"Cherry"` | Voice name string |
| `fps` | Video frames per second | `1` | Integer |
| `include_video` | Include video frames in requests | `False` | Boolean |
| `video_width` | Video frame width | `1280` | Integer |
| `video_height` | Video frame height | `720` | Integer |

## Environment Variables

Set `DASHSCOPE_API_KEY` in your environment or `.env` file:

```bash
DASHSCOPE_API_KEY=your_dashscope_api_key_here
```

## Example

See `plugins/qwen/example/qwen_realtime_example.py` for a complete working example.

## Dependencies

- vision-agents
- websockets
- aiortc
- av
vision-agents-plugins-fast-whisper 0.3.6
Fast Whisper STT integration for Vision Agents
Requires Python: >=3.10
Keywords: AI, STT, agents, faster-whisper, speech-to-text, voice agents, whisper

# Fast Whisper STT Plugin

Fast Whisper STT plugin for Vision Agents, providing real-time audio transcription using [faster-whisper](https://github.com/guillaumekln/faster-whisper).

## Features

- Fast inference using CTranslate2-based Whisper implementation
- Support for multiple model sizes (tiny, base, small, medium, large, large-v2, large-v3)
- Automatic language detection or manual language specification
- CPU and GPU support
- Quantization support (int8, float16, float32)

## Installation

```bash
uv add vision-agents[fast-whisper]
```
vision-agents 0.3.6
Open video agents. Build low latency video and voice agents on any realtime edge network.
Requires Python: >=3.10
Keywords: AI, agents, video AI, video agents, voice AI, voice agents

# Open Vision Agents by Stream

Build Vision Agents quickly with any model or video provider.

-  **Video AI**: Built for real-time video AI. Combine Yolo, Roboflow and others with gemini/openai realtime
-  **Low Latency**: Join quickly (500ms) and low audio/video latency (30ms)
-  **Open**: Built by Stream, but use any video edge network that you like
-  **Native APIs**: Native SDK methods from OpenAI (create response), Gemini (generate) and Claude (create message). So you're never behind on the latest features
-  **SDKs**: SDKs for React, Android, iOS, Flutter, React, React Native and Unity.

Created by Stream, uses [Stream's edge network](https://getstream.io/video/) for ultra-low latency.

See [Github](https://github.com/GetStream/Vision-Agents).
vision-agents-plugins-huggingface 0.3.6
HuggingFace Inference integration for Vision Agents
Requires Python: >=3.10
Keywords: AI, LLM, agents, huggingface, inference, voice agents

# HuggingFace Plugin for Vision Agents

HuggingFace Inference integration for Vision Agents. Supports both text-only LLM and vision language models (VLM) through HuggingFace's Inference Providers API.

## Installation

```bash
uv add vision-agents[huggingface]
```

## Configuration

Set your HuggingFace API token:

```bash
export HF_TOKEN=your_huggingface_token
```

## Usage

### Text-only LLM

```python
from vision_agents.plugins import huggingface

llm = huggingface.LLM(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    provider="together",  # optional: use "auto" or omit to let HuggingFace auto-select based on your settings
)

response = await llm.simple_response("Hello, how are you?")
print(response.text)
```

### Vision Language Model (VLM)

```python
from vision_agents.plugins import huggingface

vlm = huggingface.VLM(
    model="Qwen/Qwen2-VL-7B-Instruct",
    fps=1,
    frame_buffer_seconds=10,
)

# VLM automatically buffers video frames when used with an Agent
response = await vlm.simple_response("What do you see?")
print(response.text)
```

### With Function Calling

```python
from vision_agents.plugins import huggingface

llm = huggingface.LLM(model="meta-llama/Meta-Llama-3-8B-Instruct")

@llm.register_function()
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is sunny."

response = await llm.simple_response("What's the weather in Paris?")
```

## Supported Providers

HuggingFace's Inference Providers API supports multiple backends:

- Together AI
- Groq
- Cerebras
- Replicate
- Fireworks
- And more

Specify a provider explicitly or let HuggingFace auto-select:

```python
llm = huggingface.LLM(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    provider="groq",
)
```

## API Reference

### `huggingface.LLM`

Text-only language model integration.

**Parameters:**
- `model` (str): HuggingFace model ID
- `api_key` (str, optional): HuggingFace API token (defaults to `HF_TOKEN` env var)
- `provider` (str, optional): Inference provider name

### `huggingface.VLM`

Vision language model integration with video frame buffering.

**Parameters:**
- `model` (str): HuggingFace model ID
- `api_key` (str, optional): HuggingFace API token (defaults to `HF_TOKEN` env var)
- `provider` (str, optional): Inference provider name
- `fps` (int): Frames per second to buffer (default: 1)
- `frame_buffer_seconds` (int): Seconds of video to buffer (default: 10)
vision-agents-plugins-ultralytics 0.3.6
Ultralytics YOLO plugin for GetStream
Requires Python: >=3.10

# Ultralytics Plugin

This plugin provides YOLO-based pose detection capabilities for vision-agents using the Ultralytics YOLO library.

## Features

- Real-time pose detection using YOLO models
- Hand and wrist tracking with detailed connections
- Video processing with pose annotations
- Configurable confidence thresholds and processing intervals
- Support for both CPU and GPU inference

## Installation

```bash
pip install vision-agents-plugins-ultralytics
```

## Usage

```python
from vision_agents.plugins import ultralytics

# Create a YOLO pose processor
processor = ultralytics.YOLOPoseProcessor(
    model_path="yolo11n-pose.pt",
    conf_threshold=0.5,
    device="cpu",
    enable_hand_tracking=True,
    enable_wrist_highlights=True
)
```

## Configuration

- `model_path`: Path to YOLO pose model file (default: "yolo11n-pose.pt")
- `conf_threshold`: Confidence threshold for pose detection (default: 0.5)
- `imgsz`: Image size for YOLO inference (default: 512)
- `device`: Device to run inference on ('cpu' or 'cuda')
- `max_workers`: Number of worker threads for processing (default: 2)
- `interval`: Processing interval in seconds (0 for every frame)
- `enable_hand_tracking`: Whether to draw detailed hand connections
- `enable_wrist_highlights`: Whether to highlight wrist positions

## Dependencies

- ultralytics>=8.0.0
- opencv-python>=4.8.0
- numpy>=1.24.0
- pillow>=10.0.0
- aiortc>=1.6.0
- av>=10.0.0
vision-agents-plugins-turbopuffer 0.3.6
TurboPuffer RAG integration for Vision Agents with hybrid search
Requires Python: >=3.10
Keywords: AI, RAG, agents, hybrid-search, turbopuffer, vector-search, voice agents

# TurboPuffer RAG Plugin

Hybrid search RAG (Retrieval Augmented Generation) implementation using TurboPuffer for vector + BM25 full-text search, with Gemini for embeddings.

## Features

- **Hybrid Search**: Combines vector (semantic) and BM25 (keyword) search for better retrieval quality
- **Reciprocal Rank Fusion**: Merges results from multiple search strategies
- **Gemini Embeddings**: Uses Google's Gemini embedding model for high-quality vectors
- **Low-latency Queries**: Supports cache warming for fast query responses
- **Implements RAG Interface**: Compatible with Vision Agents RAG base class

## Installation

```bash
uv add vision-agents[turbopuffer]
```

## Usage

```python
from vision_agents.plugins import turbopuffer

# Initialize RAG
rag = turbopuffer.TurboPufferRAG(namespace="my-knowledge")
await rag.add_directory("./knowledge")

# Hybrid search (default)
results = await rag.search("How does the chat API work?")

# Vector-only search
results = await rag.search("How does the chat API work?", mode="vector")

# BM25-only search  
results = await rag.search("chat API pricing", mode="bm25")

# Or use convenience function
rag = await turbopuffer.create_rag(
    namespace="product-knowledge",
    knowledge_dir="./knowledge"
)
```

## Configuration

| Parameter | Description | Default |
|-----------|-------------|---------|
| `namespace` | TurboPuffer namespace for storing vectors | Required |
| `embedding_model` | Gemini embedding model | `models/gemini-embedding-001` |
| `chunk_size` | Size of text chunks for splitting documents | `10000` |
| `chunk_overlap` | Overlap between chunks for context continuity | `200` |
| `region` | TurboPuffer region | `gcp-us-central1` |

## Environment Variables

- `TURBO_PUFFER_KEY`: TurboPuffer API key
- `GOOGLE_API_KEY`: Google API key (for Gemini embeddings)

## Dependencies

- `turbopuffer`: TurboPuffer vector database client
- `langchain-google-genai`: Gemini embeddings
- `langchain-text-splitters`: Text chunking utilities

## References

- [TurboPuffer Hybrid Search](https://turbopuffer.com/docs/hybrid)
- [MTEB Embedding Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
kiber-chat 0.0.1
Kiber.chat - Matrix-based unified messaging platform (coming soon)
Requires Python: >=3.10
Keywords: bridges, chat, element, matrix, messaging, telegram, whatsapp

# kiber-chat

**Kiber.chat** - Matrix-based unified messaging platform.

## Coming Soon

This package is under active development. Features planned:

- Matrix protocol client
- Bridges to Telegram, WhatsApp, and other messengers
- Unified inbox for all your communications
- End-to-end encryption

## Links

- Website: https://kiber.chat
- Matrix server: kiber.chat

## Installation

```bash
pip install kiber-chat
```

*Note: This is a placeholder package. Full functionality coming soon.*

## License

Apache 2.0
libsightseeing 0.2.0
a shared library for file finding and source resolution with gitignore support
Requires Python: >=3.11
Keywords: file-finding, gitignore, source-resolution

# libsightseeing

a shared library for file finding and source resolution with gitignore support,
after i needed a simple way to find files while respecting .gitignore

## installation

```text
pip install libsightseeing
```

## usage

### finding project root

```python
from libsightseeing import find_project_root

# find project root from current directory
root = find_project_root()
if root:
    print(f"found project at: {root}")

# find from a subdirectory
root = find_project_root("~/Works/example/sub/dir")
if root:
    print(f"found project at: {root}")  # ~/Works/example

# use with custom markers
root = find_project_root(
    ".",
    markers=[".git", "pyproject.toml", "package.json"]
)
```

### finding files (simple)

```python
from libsightseeing import find_files

# find all python files
files = find_files(".", include=["*.py"])

# find files excluding tests
files = find_files("src", exclude=["tests"])

# include gitignored files
files = find_files(".", include=["*.py"], respect_gitignore=False)
```

### finding files (advanced)

use `SourceResolver` when you need more control or want to reuse the resolver:

```python
from libsightseeing import SourceResolver

# create a reusable resolver
resolver = SourceResolver(
    root=".",
    include=["src/**/*.py"],
    exclude=["tests", "*.pyc"],
    respect_gitignore=True,
)

# resolve files multiple times (e.g., in a watch loop)
files = resolver.resolve()
```

### combining both

```python
from libsightseeing import find_project_root, find_files

# find project root first, then find files there
root = find_project_root(".")
if root:
    files = find_files(root, include=["*.py"])
```

## api differences

**`find_files()` vs `SourceResolver`**

- **`find_files()`** — simple one-liner function. creates a resolver internally
  and returns files immediately. use this for one-off file finding.

- **`SourceResolver`** — class-based api. gives you a reusable resolver object
  that you can call `.resolve()` on multiple times. use this when you need to
  search the same directory repeatedly or want more control.

both respect .gitignore and support the same include/exclude patterns.

## features

- **find project root**  
  walk up the tree looking for .git, pyproject.toml, package.json, cargo.toml, etc.

- **respects .gitignore**  
  automatically excludes gitignored files

- **glob patterns**  
  supports include/exclude patterns

- **simple one-liner**  
  `find_files()` and `find_project_root()` for quick usage

- **configurable resolver**  
  `SourceResolver` for advanced cases

- **lightweight**  
  only depends on pathspec

## licence

libsightseeing is unencumbered, free-as-in-freedom, and is dual-licenced under
The Unlicense or the BSD Zero Clause License. (SPDX: `Unlicense OR 0BSD`)

you are free to use the software as you wish, without any restrictions or
obligations, subject only to the warranty disclaimers in the licence text
of your choosing.
pvfalcon 2.0.1
Falcon Speaker Diarization Engine
Requires Python: >=3.9
Keywords: Speaker Diarization, Speaker Identification, Voice Identification

# Falcon Binding for Python

## Falcon Speaker Diarization Engine

Made in Vancouver, Canada by [Picovoice](https://picovoice.ai)

Falcon is an on-device speaker diarization engine. Falcon is:

- Private; All voice processing runs locally.
- Cross-Platform:
  - Linux (x86_64), macOS (x86_64, arm64), Windows (x86_64, arm64)
  - Raspberry Pi (3, 4, 5)

## Compatibility

- Python 3.9+
- Runs on Linux (x86_64), macOS (x86_64, arm64), Windows (x86_64, arm64), and Raspberry Pi (3, 4, 5).

## Installation

```console
pip3 install pvfalcon
```

## AccessKey

Falcon requires a valid Picovoice `AccessKey` at initialization. `AccessKey` acts as your credentials when using Falcon SDKs.
You can get your `AccessKey` for free. Make sure to keep your `AccessKey` secret.
Signup or Login to [Picovoice Console](https://console.picovoice.ai/) to get your `AccessKey`.

### Usage

Create an instance of the engine and perform speaker diarization on an audio file:

```python
import pvfalcon

handle = pvfalcon.create(access_key='${ACCESS_KEY}')

segments = handle.process_file('${AUDIO_PATH}')
for segment in segments:
    print("{speaker tag=%d - start_sec=%.2f end_sec=%.2f}"
          % (segment.speaker_tag, segment.start_sec, segment.end_sec))
```

Replace `${ACCESS_KEY}` with yours obtained from [Picovoice Console](https://console.picovoice.ai/) and
`${AUDIO_PATH}` to the path an audio file. Finally, when done be sure to explicitly release the resources using
`handle.delete()`.

## Demos

[pvfalcondemo](https://pypi.org/project/pvfalcondemo/) provides command-line utilities for processing audio using
Falcon.
iflow-mcp_fr0gger-orkl 0.1.0
A Model Context Protocol (MCP) server for querying the ORKL API
Requires Python: >=3.11

# ORKL MCP Server

![MIT License](https://img.shields.io/badge/license-MIT-green)

A full write-up can be found here: https://blog.securitybreak.io/building-a-threat-intelligence-genai-reporter-with-orkl-and-claude-a0ae2e969693

A Model Context Protocol (MCP) server for querying the ORKL API. This server provides tools for fetching and analyzing threat reports, threat actors, and sources. It integrates smoothly with MCP-compatible applications.

---
# Quick Install
Edit or create the file /Users/user/Library/Application Support/Claude/claude_desktop_config.json
```
{
  "mcpServers": {
    "orkl": {
      "command": "uv",
      "args": [
      "--directory",
      "/MyMCP/mcptest/orkl",
      "run",
      "orkl"
      ]
    }
  }
}
```
---

# Tools

## Report Tools

### Fetch Latest Threat Reports
- **Name**: `fetch_latest_threat_reports`
- **Description**: Fetch recent threat reports with their titles and IDs.
- **Parameters**: None

### Fetch Threat Report Details
- **Name**: `fetch_threat_report_details`
- **Description**: Retrieve detailed information for a specific threat report by ID.
- **Parameters**:
  - `report_id` (required): The ID of the threat report.

## Threat Actor Tools

### Fetch Threat Actors
- **Name**: `fetch_threat_actors`
- **Description**: Fetch a list of known threat actors with their IDs and names.
- **Parameters**: None

### Fetch Threat Actor Details
- **Name**: `fetch_threat_actor_details`
- **Description**: Retrieve detailed information for a specific threat actor by ID.
- **Parameters**:
  - `actor_id` (required): The ID of the threat actor.

## Source Tools

### Fetch Sources
- **Name**: `fetch_sources`
- **Description**: Fetch a list of sources used in threat intelligence.
- **Parameters**: None

### Fetch Source Details
- **Name**: `fetch_source_details`
- **Description**: Retrieve detailed metadata for a specific source by ID.
- **Parameters**:
  - `source_id` (required): The ID of the source.
fw-gear-dicom-qc 0.5.2
Validate dicom archive on a set of hardcoded and user-specified rules
Requires Python: <4,>=3.11
Keywords: AEQC, DICOM, Flywheel, Gears, QC

# DICOM QC gear

dicom-qc is a Flywheel Gear that evaluates basic QC rules against a dicom archive.

## Documentation

[Gear Documentation](https://flywheel-io.gitlab.io/scientific-solutions/gears/dicom-qc/index.html).

## Contributing

Please refer to the CONTRIBUTING.md file for information on how to
contribute to the gear.