semvec/pypi-packages huggingface dataset:
delta-fusion 1.0.3
High-performance Delta Lake query engine for Python
License: MIT
Requires Python: >=3.11
Keywords: delta-lake, datafusion, arrow, parquet, data-engineering, query-engine, polars, pyarrow

# deltaFusion

High-performance Delta Lake query engine for Python, powered by Rust.

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Overview

deltaFusion provides SQL query capabilities over Delta Lake tables using [DataFusion](https://github.com/apache/datafusion) as the query engine. Data is transferred via zero-copy [Apache Arrow](https://arrow.apache.org/) for maximum performance.

### Key Features

- **SQL Queries**: Full SQL support via DataFusion
- **Zero-Copy Transfer**: Arrow-based data exchange with Python (no serialization overhead)
- **Time Travel**: Query specific table versions
- **Storage Support**: Local filesystem and S3-compatible storage
- **GIL Release**: Python threads remain active during Rust operations

## Installation

```bash
# From source (requires Rust toolchain and maturin)
pip install maturin
maturin develop --release
```

## Quick Start

```python
from delta_fusion import DeltaEngine

# Create engine
engine = DeltaEngine()

# Register a Delta table
engine.register_table("sales", "/path/to/delta/table")

# Query with SQL (returns PyArrow RecordBatches)
batches = engine.query("SELECT * FROM sales WHERE year = 2024")

# Convert to pandas
import pyarrow as pa
table = pa.Table.from_batches(batches)
df = table.to_pandas()

# Or get results as list of dicts (for small datasets)
rows = engine.query_to_dicts("SELECT * FROM sales LIMIT 10")
```

### S3 Storage

```python
# Option 1: Environment variables
# AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION

# Option 2: Explicit configuration
engine = DeltaEngine({
    "aws_access_key_id": "...",
    "aws_secret_access_key": "...",
    "aws_region": "us-east-1",
    "aws_endpoint": "http://localhost:9000",  # For MinIO
    "aws_allow_http": True
})

engine.register_table("data", "s3://bucket/delta-table")
```

### Time Travel

```python
# Query a specific version
engine.register_table("sales", "/path/to/table", version=5)

# Get table metadata
info = engine.table_info("/path/to/table")
print(f"Version: {info['version']}, Files: {info['num_files']}")
```

## Development

```bash
# Build for development
maturin develop

# Run Rust tests
cargo test --no-default-features

# Run Python tests
pytest

# Check and format
cargo check
cargo clippy
cargo fmt
```

## Architecture

```
src/
├── lib.rs       # Module entry point
├── config.rs    # Storage configuration
├── error.rs     # Error types
├── engine.rs    # DeltaEngine (DataFusion integration)
└── python.rs    # PyO3 bindings

python/delta_fusion/
├── __init__.py  # Python API
└── __init__.pyi # Type stubs
```

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
pipen-log2file 1.1.0
Add verbosal information in logs for pipen.
License: MIT
Requires Python: <4.0,>=3.9

# pipen-log2file

Save running logs to file for [pipen][1].

The log file is saved to `<workdir>/<pipeline>/.logs/run-<date-time>.log` by default.
A symlink `<workdir>/<pipeline>/run-latest.log` is created to the latest log file.

The xqute logs are also saved to `<workdir>/<pipeline>/<proc>/proc.xqute.log`

Note that the original handler of xqute logger is removed during pipeline running.

## Options

- `plugin_opts.log2file_xqute`: Whether to save xqute logs. Default: `True`.
    if False, the xqute logger will be kept intact.
- `plugin_opts.log2file_xqute_level`: The log level for xqute logger. Default: `INFO`.
- `plugin_opts.log2file_xqute_append`: Whether to append to the log file. Default: `False`.

## Installation

```
pip install -U pipen-log2file
```

## Enabling/Disabling the plugin

The plugin is registered via entrypoints. It's by default enabled. To disable it:
`plugins=[..., "no:log2file"]`, or uninstall this plugin.


[1]: https://github.com/pwwang/pipen
mypy-11-ren 0.0.3
A small python package
License: MIT
Requires Python: >=3.8


# mypy_11_ren

This repository contains the source code and documentation for the
**Inception Renderer** Python package, available on PyPI.

The Inception Renderer is a tool for rendering YouTube videos and web pages
inside **Jupyter Notebook** and **Google Colab** environments.

---

## 🚀 How to Run

### 📦 Installation

#### 1. Create a conda environment

conda create -n inrenderer python=3.8 -y

#### 2. Activate the environment

<pre class="overflow-visible! px-0!" data-start="650" data-end="687"><div class="contain-inline-size rounded-2xl corner-superellipse/1.1 relative bg-token-sidebar-surface-primary"><div class="sticky top-[calc(--spacing(9)+var(--header-height))] @w-xl/main:top-9"><div class="absolute end-0 bottom-0 flex h-9 items-center pe-2"><div class="bg-token-bg-elevated-secondary text-token-text-secondary flex items-center gap-4 rounded-sm px-2 font-sans text-xs"></div></div></div><div class="overflow-y-auto p-4" dir="ltr"><code class="whitespace-pre! language-bash"><span><span>conda activate inrenderer
</span></span></code></div></div></pre>

#### 3. Install dependencies

pip install -r requirements_dev.txt