pipreq-distill 0.1.0
Resolve pip dependency conflicts by intelligently removing problematic packages
Requires Python: >=3.11
Keywords: conflicts, dependencies, pip, pyproject, requirements, resolver

# pipreq-distill

Resolve pip dependency conflicts by intelligently removing problematic packages.

When `pip install` fails with dependency conflicts, **pipreq-distill** queries PyPI to find the largest possible collection of compatible packages by removing the oldest and most problematic dependencies.

## Installation

```bash
pip install pipreq-distill
```

Or with uv:

```bash
uv pip install pipreq-distill
```

## Usage

### Basic Usage

```bash
# From requirements.txt to resolved.txt
pipreq-distill --input requirements.txt --output resolved.txt

# From stdin to stdout
cat requirements.txt | pipreq-distill > resolved.txt

# Dry run (show what would be removed)
pipreq-distill --input requirements.txt --dry-run
```

### pyproject.toml Support

```bash
# Process pyproject.toml dependencies
pipreq-distill --input pyproject.toml --output pyproject.toml --format pyproject

# Process optional dependencies
pipreq-distill --input pyproject.toml --group optional.dev --dry-run

# Update in-place
pipreq-distill --input pyproject.toml --output pyproject.toml
```

### Protect Specific Packages

```bash
# Never remove requests or flask
pipreq-distill --input requirements.txt --keep "requests,flask"
```

### JSON Output

```bash
# Get structured output for scripting
pipreq-distill --input requirements.txt --json-output
```

Output:
```json
{
  "kept": ["flask==2.3.0", "requests==2.31.0"],
  "removed": [
    {"name": "old-pkg", "version": "1.0.0", "score": 150.5}
  ],
  "kept_count": 2,
  "removed_count": 1
}
```

## How It Works

1. **Fetches package metadata** from PyPI (with caching)
2. **Extracts dependencies** from wheels or source distributions
3. **Detects version conflicts** between packages
4. **Scores packages** for removal based on:
   - Conflict involvement (packages in more conflicts score higher)
   - Reverse dependencies (packages with fewer dependents score higher)
   - Package age (older packages score higher)
   - Dependency age (packages depending on old packages score higher)
5. **Iteratively removes** the highest-scoring package until no conflicts remain
6. **Outputs** the largest possible set of compatible packages

## CLI Options

| Option | Description | Default |
|--------|-------------|---------|
| `--input` | Input file (requirements.txt or pyproject.toml) | stdin |
| `--output` | Output file | stdout |
| `--format` | Format: `auto`, `requirements`, `pyproject` | `auto` |
| `--group` | Dependency group for pyproject.toml | `dependencies` |
| `--keep` | Comma-separated packages to protect | None |
| `--dry-run` | Show what would be removed | False |
| `--json-output` | Output as JSON | False |
| `--verbose` | Show detailed progress | False |
| `--timeout` | PyPI request timeout (seconds) | 30 |
| `--max-concurrent` | Max concurrent PyPI requests | 20 |

## Example

Given conflicting requirements:

```
aliyun-python-sdk-core>=2.16.0
jmespath>=1.0.1
```

Running:

```bash
$ pipreq-distill --input requirements.txt --verbose

Parsed 2 requirements
Fetching 2 packages in parallel...
Fetched 2 packages successfully
Iteration 1: 1 conflicts in 1 cluster(s)
Removing aliyun-python-sdk-core (score: 641.4)
Resolved to 1 compatible packages

jmespath==1.0.1
```

The tool identified that `aliyun-python-sdk-core` requires `jmespath<1.0.0`, which conflicts with `jmespath>=1.0.1`. Since `aliyun-python-sdk-core` is older and has the conflicting requirement, it was removed.

## Dependency Groups

For pyproject.toml, you can specify different dependency groups:

- `dependencies` - PEP 621 project.dependencies (default)
- `optional.dev` - project.optional-dependencies.dev
- `optional.test` - project.optional-dependencies.test
- `dev` - tool.uv.dev-dependencies

## Caching

PyPI responses are cached in `$TMPDIR/pipreq_distill_cache` for 24 hours to speed up repeated runs.

## Limitations

- Assumes you want the latest compatible versions
- Conditional dependencies (platform-specific markers) are simplified
- Uses a greedy algorithm (not guaranteed globally optimal)
- May download wheels/sdists for packages missing JSON metadata

## License

Apache-2.0