semvec/pypi-packages huggingface dataset:
mpmsub 2025.10.16
Memory-aware multiprocessing subprocess execution library
License: MIT
Requires Python: >=3.8
Keywords: multiprocessing, memory, subprocess, parallel, scheduling

# mpmsub - Memory-Aware Multiprocessing Subprocess

A Python library for running subprocess commands with intelligent memory-aware scheduling and resource management.

## Features

- Memory-aware scheduling with automatic resource management
- Pipeline support for chaining commands
- Multiple job interfaces (dictionary, object-oriented, convenience functions)
- Output redirection and progress tracking
- Job profiling for memory optimization
- Flexible API with multiple parameter names

## Installation

```bash
pip install mpmsub
```

## Quick Start

```python
import mpmsub

# Create cluster and add jobs
p = mpmsub.cluster(cpu=4, memory="8G")

# Dictionary interface
p.jobs.append({"cmd": ["echo", "hello"], "p": 1, "m": "1G"})

# Object interface
p.jobs.append(mpmsub.Job(["python", "script.py"]).cpu(2).memory("2G"))

# Pipeline interface
p.jobs.append(mpmsub.pipeline([
    ["cat", "data.txt"],
    ["grep", "pattern"]
], cpu=1, memory="500M"))

# Run and analyze
results = p.run()
print(f"Completed: {results['jobs']['completed']}/{results['jobs']['total']}")
```

## Performance

Run benchmarks to see performance benefits:

```bash
python examples/benchmark_demo.py
python benchmarks/run_all_benchmarks.py
```

Benefits include 1.2-2x speedup in memory-constrained scenarios and better system stability. See [`benchmarks/README.md`](benchmarks/README.md) for details.

## Documentation

**Complete documentation:** [https://nextgenusfs.github.io/mpmsub/](https://nextgenusfs.github.io/mpmsub/)

Includes tutorials, API reference, examples, and performance tips.

## Key Features

### Multiple Job Interfaces

```python
# Dictionary interface
p.jobs.append({"cmd": ["echo", "hello"], "p": 1, "m": "1G"})

# Object interface with builder pattern
job = mpmsub.Job(["python", "script.py"]) \
    .cpu(2).memory("4G") \
    .stdout_to("output.txt")

# Pipeline interface
pipeline = mpmsub.pipeline([
    ["cat", "data.txt"],
    ["grep", "pattern"],
    ["sort"]
], cpu=1, memory="500M")
```

### Job Profiling

```python
# Measure actual memory usage
profile_results = p.profile()

# Use recommendations for optimized scheduling
p.jobs.append({"cmd": ["my_command"], "p": 1, "m": "150M"})
```

### Memory Formats

- `"1G"` - Gigabytes
- `"1024M"` - Megabytes
- `1024` - MB (integer)

## Examples

See the `examples/` directory for usage demonstrations and the documentation for comprehensive tutorials.

## Development

```bash
# Clone and install in development mode
git clone https://github.com/nextgenusfs/mpmsub.git
cd mpmsub
pip install -e .[dev]

# Install pre-commit hooks (optional)
pre-commit install
```

### CI/CD

The project uses GitHub Actions for:
- **Tests**: Run on Python 3.8-3.12, Ubuntu/macOS
- **Code Quality**: Linting, formatting, type checking
- **Publishing**: Manual workflow for TestPyPI/PyPI, automatic PyPI on releases
- **Documentation**: Auto-deploy to GitHub Pages

## Requirements

- Python 3.8+
- psutil >= 5.8.0

## License

MIT License
PyObjectInterface 0.1.3
Create a web interface to interact with a class
License: MIT
Requires Python: >=3.10

# PyObjectInterface

![Tests Badge](https://github.com/RobertJN64/PyObjectInterface/actions/workflows/tests.yml/badge.svg)
![Python Version Badge](https://img.shields.io/pypi/pyversions/PyObjectInterface)
![License Badge](https://img.shields.io/github/license/RobertJN64/PyObjectInterface)

Quickly control any python object through an async web interface.

`pip install PyObjectInterface`
```python
from PyObjectInterface.WebController import create_WebController
import flask

app = flask.Flask('Demo')
any_python_object = ...
create_WebController(any_python_object, 'any_python_object', app)
app.run()
```

Go to `http://127.0.0.1:5000/webcontroller`

## Example: Async Robot Control
![async_robot_control.png](async_robot_control.png)

The above interface was automatically created from:
```python
class DriveMotor:
    def __init__(self, pin):
        self.pin = pin
        self.speed = 0

    def forward(self): ...
    def stop(self): ...

class Robot:
    def __init__(self):
        self.left_motor = DriveMotor(1)
        self.right_motor = DriveMotor(2)

    def forward(self): ...
    def stop(self): ...
    def left(self): ...
    def right(self): ...
```

Look at [PyObjectInterface/examples/example.py](PyObjectInterface/examples/example.py) for more information.

## Advanced Usage and Settings

You can use PyObjectInterface without the WebController.

### create_WebController
 - `obj`: Any python object
 - `name: str`: Friendly name for object, generally the name of the variable that refers to the object
 - `flask_app`: return value of flask.Flask()
 - `recursion_depth = 5`: maximum layers of subobjects to include
 - `create_private_interface = True`: if `True` creates a `/webcontroller/private` page that includes private methods and attributes

### Modifying Attributes

By default, POI treats `NoneType, bool, int, float, str, list, dict, tuple, and set` as primitive types.
These show up in the attribute table instead of as recursive objects.

`numpy.ndarray`, `pandas.DataFrame`, and `pandas.Series` are also treated as primitives.

Note: numpy 2.0 arrays currently do not work with python's inspect library, 
and therefore must be treated as a primitive.
For Python >= 3.13 on windows, numpy >= 2.0 must be used.

If you would like to customize this behavior, you can modify
`PyObjectInterface.basic_attribute_types`

For example:
```python
from PyObjectInterface import basic_attribute_types

class My_Class_A: ...
class My_Class_B: ...
class My_Class_C: ...

basic_attribute_types[str(type(My_Class_A()))] = False
basic_attribute_types[str(type(My_Class_B()))] = True
```

`My_Class_A` will be treated as a primitive and only show up in the attributes table.
`My_Class_B` will also show up in attributes table, but it still appears in the subobjects section.
`My_Class_C` will only appear in the subobjects section.