semvec/pypi-packages huggingface dataset:
video-transcriber 0.3.1
A Python project following TDD principles
Requires Python: >=3.8

# video-transcriber

Extracts visually distinct frames from videos and transcribes audio using Whisper.
Creates a portable zip file containing a markdown transcript with slide images in the img sub-directory.

## Features

- **Smart slide detection** - Uses perceptual hashing to capture only distinct frames, not every frame
- **Audio transcription** - Uses Whisper AI locally to transcribe speech to text
- **Timeline merging** - Associates transcribed audio with the corresponding slides
- **Portable output** - Generates a zip file with markdown and images that works anywhere

## Requirements

- Python 3.10+
- ffmpeg (for audio extraction)

```bash
# Ubuntu/Debian
apt install ffmpeg

# macOS
brew install ffmpeg

# Windows (using winget)
winget install ffmpeg
```

## Installation

```bash
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
uv pip install video-transcriber
```

### Hugging Face

The first time you run the transcriber, it downloads Whisper models from Hugging Face.
This may take several minutes.

You may see this warning:

```
Warning: You are sending unauthenticated requests to the HF Hub.
```

This is harmless. Everything will still work.

## Usage


### Python API

```python
from video_transcriber.transcribe import transcribe_video

zip_path = transcribe_video("my-presentation.mp4", "output/")
```

The output zip file contains:
```
output/my-presentation_transcript.zip
├── transcript.md
└── img/
    ├── frame_000.png
    ├── frame_001.png
    └── ...
```

### Options

```python
zip_path = transcribe_video(
    "my-presentation.mp4",
    "output/",
    model_size="large-v3",     # Whisper model: tiny, base, small, medium, large-v3
    sample_interval=15,        # Check for new slides every N frames (default: 30)
    include_timestamps=True,   # Include timestamps in markdown output (default: False)
    audio_only=False           # Transcribe audio only, skip frame extraction (default: False)
)
```

### Audio-Only Mode

For videos where you only need the audio transcript (podcasts, interviews, etc.):

```python
zip_path = transcribe_video(
    "podcast.mp4",
    "output/",
    audio_only=True,
    include_timestamps=True
)
```

This skips frame extraction entirely, producing a zip with just the text transcript.

## Development

```bash
git clone https://github.com/romilly/video-transcriber.git
cd video-transcriber
python -m venv venv
source venv/bin/activate
pip install -e .[test]
pytest
```

### Quick Demo

Run the demo script to process the included test video:

```bash
python demo_create_zip.py
```

This processes `tests/data/demo.mp4` and creates `tests/data/generated/demo.zip`.
rag-scrubber 0.1.4
Auto-removes headers, footers, and noise from PDF text for RAG apps.
Requires Python: >=3.7

# rag-scrubber

**Clean your "dirty" PDF text before feeding it to LLMs.**

`rag-scrubber` is a lightweight Python library designed for **RAG (Retrieval-Augmented Generation)** pipelines. It automatically detects and removes recurring headers, footers, and page numbers from extracted text, and fixes broken hyphenation.

## 😟 The Problem

When extracting text from PDFs (especially corporate reports or scanned docs), you often get artifacts that confuse LLMs:

> *"The Q3 revenue was **CONFIDENTIAL REPORT 2024** higher than expected..."*

These interruptions waste tokens and degrade model performance. `rag-scrubber` cleans this mess up.

## ✨ Features

* **Auto-Header/Footer Removal:** Uses positional statistics to detect lines that repeat across pages (e.g., "Page 1 of 50", "Confidential").
* **Smart De-Hyphenation:** Fixes words split across lines (e.g., `com-` + `puter`  `computer`) without merging proper nouns.
* **Zero-Shot Configuration:** Works out of the box with sensible defaults, or tune the sensitivity yourself.
* **Lightweight:** No heavy ML dependencies. pure Python logic.

## 📦 Installation

```bash
pip install rag-scrubber

```

## 🚀 Quick Start

```python
from rag_scrubber import RAGScrubber

# 1. Simulate dirty text (usually coming from PyPDF2 or PDFMiner)
pages = [
    "Q3 FINANCIAL REPORT - CONFIDENTIAL\nThe revenue increased by 20%.\nPage 1",
    "Q3 FINANCIAL REPORT - CONFIDENTIAL\nOperating costs were lowered.\nPage 2",
    "Q3 FINANCIAL REPORT - CONFIDENTIAL\nNew user acquisition is sta-\nble.\nPage 3"
]

# 2. Initialize the scrubber
# threshold=0.4 means if a line appears in >40% of pages, it's garbage.
scrubber = RAGScrubber(threshold=0.4)

# 3. Clean the text
clean_text = scrubber.clean(pages)

print(clean_text)

```

**Output:**

```text
The revenue increased by 20%.

Operating costs were lowered.

New user acquisition is stable.

```

## ⚙️ Advanced Usage

You can adjust the sensitivity if it is deleting too much (or too little).

```python
# Low threshold (0.1) = Very aggressive (removes anything that repeats even slightly)
# High threshold (0.9) = Very safe (only removes exact matches on almost every page)
scrubber = RAGScrubber(threshold=0.2) 

```

## 🤝 Contributing

1. Fork the repository.
2. Create a feature branch (`git checkout -b feature/new-logic`).
3. Commit your changes.
4. Push to the branch.
5. Open a Pull Request.

## 📄 License

MIT License. Free to use for commercial and personal projects.
zampie_utils 0.9.1
A collection of utility functions
Requires Python: >=3.6
Keywords: utils, tools, utilities

# Zampie Utils

一个实用的Python工具包，提供了一系列常用的工具函数。

## 功能特性

- 文件操作工具
- 计时器上下文管理器
- 表情符号和表情识别
- 概率触发器
- 会话保存功能
- 智能日志系统（支持自动滚动日志）

## 安装

```bash
pip install zampie_utils
```

## 使用示例

```python
from zampie_utils import ContextTimer, read_file
from zampie_utils.logger import Logger

# 使用计时器
with ContextTimer("my_task", "s"):
    # 你的代码
    pass

# 读取文件
content = read_file("path/to/your/file.txt")

# 使用滚动日志
logger = Logger()
# 添加文件处理器，设置最大文件大小为10MB，保留5个备份文件
logger.add_file_handler("app.log", max_bytes=10*1024*1024, backup_count=5)
logger.info("这是一条日志消息")
```

## PyPi

https://pypi.org/project/zampie-utils/