semvec/pypi-packages huggingface dataset:
delta-lake-utils 1.0.0
Production-grade utilities for Delta Lake management
Requires Python: >=3.8

# delta-lake-utils

Production-grade utilities for Delta Lake table management and optimization on Databricks.

## What Does This Package Do?

Automates Delta Lake table optimization, health monitoring, and pipeline generation for Databricks data engineers.

### Main Features:

1. **Smart OPTIMIZE** - Automatically consolidates small files and improves query performance
   - Detects when your Delta table has too many small files
   - Intelligently chooses which columns to Z-ORDER by
   - Reduces query time by up to 10x

2. **Health Checker** - Diagnoses table problems before they impact production
   - Identifies small file problems
   - Detects data skew across partitions
   - Finds configuration issues

3. **Performance Profiler** - Measures how fast your Delta operations run
   - Track read/write speeds
   - Identify bottlenecks
   - Compare before/after optimization

4. **Medallion Generator** - Auto-creates Bronze/Silver/Gold pipeline code
   - Generates production-ready notebooks
   - Follows best practices
   - Saves hours of boilerplate coding

5. **Unity Catalog Auditor** - Manages permissions and access control
   - Audits table permissions
   - Generates permission scripts
   - Ensures security compliance

## Installation
```bash
pip install delta-lake-utils
```

## Quick Start
```python
from pyspark.sql import SparkSession
from delta_utils import DeltaOptimizer

spark = SparkSession.builder.getOrCreate()
optimizer = DeltaOptimizer(spark)

# Optimize a table - reduces files, improves performance
result = optimizer.auto_optimize('/mnt/delta/my_table')
print(f"Optimized! Removed {result.files_removed} files")
```

## Use Cases

### Use Case 1: Your queries are slow
Problem: Delta table has 5000 small files, queries take 10 minutes  
Solution: Run optimizer, consolidates to 50 files, queries now take 1 minute

### Use Case 2: Starting a new data pipeline
Problem: Need to build Bronze/Silver/Gold architecture from scratch  
Solution: Use medallion generator, get complete pipeline in 30 seconds

### Use Case 3: Data quality issues
Problem: Not sure if table is healthy, production keeps failing  
Solution: Run health checker, get specific recommendations to fix issues

### Use Case 4: Permission audit required
Problem: Need to verify all tables have correct access controls  
Solution: Use catalog auditor to check and fix permissions

## Documentation

- See [QUICKSTART.md](QUICKSTART.md) for 5-minute tutorial
- See [EXPLANATION.md](EXPLANATION.md) for detailed use cases
- See [examples/EXAMPLES.md](examples/EXAMPLES.md) for code examples

## Requirements

- Python 3.8+
- PySpark 3.2+
- Delta Lake 2.0+
- Databricks Runtime 11.0+ recommended

## Author

Nalini Panwar
GitHub: [@panwarnalini-hub](https://github.com/panwarnalini-hub)

## License

MIT License - see [LICENSE](LICENSE) file for details
pycares 5.0.1
Python interface for c-ares
Requires Python: >=3.9

pycares: Python interface for c-ares
====================================

pycares is a Python module which provides an interface to c-ares.
`c-ares <https://c-ares.org>`_ is a C library that performs
DNS requests and name resolutions asynchronously.


Documentation
-------------

http://readthedocs.org/docs/pycares/


Bundled c-ares
--------------

pycares currently bundles c-ares as a submodule for ease of building. Using the system
provided c-ares is possible if the ``PYCARES_USE_SYSTEM_LIB`` environment variable is
set to ``1`` when building.

NOTE: Versions prior to 4.0.0 used to embed a modified c-ares with extended TTL support.
That is no longer the case and as a result only A and AAAA records will have TTL information.
Follow this PR in uppstream c-ares, looks like TTLs will be added: https://github.com/c-ares/c-ares/pull/393


Installation
------------

GNU/Linux, macOS, Windows, others:

::

    pip install pycares

FreeBSD:

::

    cd /usr/ports/dns/py-pycares && make install


IDNA 2008 support
^^^^^^^^^^^^^^^^^

If the ``idna`` package is installed, pycares will support IDNA 2008 encoding otherwise the builtin idna codec will be used,
which provides IDNA 2003 support.

You can force this at installation time as follows:

::

   pip install pycares[idna]


Running the test suite
----------------------

From the top level directory, run: ``python -m unittest -v``

NOTE: Running the tests requires internet access and are somewhat environment sensitive because real DNS queries
are made, there is no mocking. If you observe a failure that the CI cannot reproduce, please try to setup an
environment as close as the current CI.


Using it from the cli, a la dig
-------------------------------

This module can be used directly from the command line in a similar fashion to dig (limited, of course):

::

   $ python -m pycares google.com
   ;; QUESTION SECTION:
   ;google.com			IN	A

   ;; ANSWER SECTION:
   google.com		300	IN	A	172.217.17.142

   $ python -m pycares mx google.com
   ;; QUESTION SECTION:
   ;google.com			IN	MX

   ;; ANSWER SECTION:
   google.com		600	IN	MX	50 alt4.aspmx.l.google.com
   google.com		600	IN	MX	10 aspmx.l.google.com
   google.com		600	IN	MX	40 alt3.aspmx.l.google.com
   google.com		600	IN	MX	20 alt1.aspmx.l.google.com
   google.com		600	IN	MX	30 alt2.aspmx.l.google.com


Author
------

Saúl Ibarra Corretgé <s@saghul.net>


License
-------

Unless stated otherwise on-file pycares uses the MIT license, check LICENSE file.


Supported Python versions
-------------------------

Python >= 3.9 are supported. Both CPython and PyPy are supported.


Contributing
------------

If you'd like to contribute, fork the project, make a patch and send a pull
request. Have a look at the surrounding code and please, make yours look
alike.
complexipy-mcp 0.1.1
A Model Context Protocol server for cognitive complexity analysis using complexipy
Requires Python: >=3.11

# Complexipy MCP Server

A Model Context Protocol (MCP) server that wraps `complexipy` to provide cognitive complexity analysis for Python codebases. This server uses the `complexipy` Python API for accurate and robust analysis.

## Features

- **Directory Scanning**: Recursively analyze all `.py` files in a directory and filter by complexity threshold.
- **File Scanning**: Analyze a single Python file.
- **Data Processing**: Uses `pandas` for efficient data filtering and sorting.
- **Clean Output**: Returns JSON formatted results (list of objects).
- **No Emojis**: Pure data output suitable for machine consumption.
- **Stdio Transport**: Uses standard input/output for communication, making it compatible with most MCP clients.

## Installation & Usage

### Quick Start

You can run the server directly using `uvx` (no installation required):

```bash
uvx complexipy-mcp
```

### Configuration for Claude Desktop

Add the following configuration to your MCP client settings (e.g., `claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "complexipy": {
      "command": "uvx",
      "args": [
        "complexipy-mcp"
      ]
    }
  }
}
```

### Local Development

1. Clone the repository.
2. Install dependencies:
   ```bash
   uv sync
   ```
3. Run the server locally:
   ```bash
   uv run complexipy-mcp
   ```

#### Local Configuration for Claude Desktop

If you are developing locally, point the configuration to your local setup:

```json
{
  "mcpServers": {
    "complexipy": {
      "command": "uv",
      "args": [
        "--directory",
        "/ABSOLUTE/PATH/TO/complexipy-mcp",
        "run",
        "complexipy-mcp"
      ]
    }
  }
}
```

## Tools

### `scan_directory`
Scans a directory for files exceeding a cognitive complexity threshold.

- **Arguments**:
  - `directory_path` (string): Absolute path to the directory to scan.
  - `threshold` (integer): Complexity limit (default: 15).
- **Returns**: JSON string containing a list of files that exceed the threshold.
  ```json
  [
    {
      "file_path": "/abs/path/to/file.py",
      "complexity": 25,
      "functions": [
        {
          "name": "function_name",
          "complexity": 10,
          "line_start": 5,
          "line_end": 15
        }
      ],
      "status": "exceeds_threshold"
    }
  ]
  ```

### `scan_file`
Scans a single file for cognitive complexity.

- **Arguments**:
  - `file_path` (string): Absolute path to the file.
  - `threshold` (integer): Complexity limit (default: 15).
- **Returns**: JSON string containing the result if it exceeds the threshold.