pysysmetrics 0.1.4
CLI tool to monitor CPU and GPU memory usage of Python processes
Requires Python: >=3.7

# PySysMetrics

**Real-time monitoring for CPU and GPU memory usage of Python processes.**

PySysMetrics is a lightweight command-line tool and Python library that provides real-time monitoring of system resources consumed by Python processes. Track CPU memory, GPU memory (NVIDIA), CPU utilization, and thread count - perfect for profiling machine learning workflows, debugging memory leaks, and optimizing resource-intensive applications.

[![PyPI version](https://badge.fury.io/py/pysysmetrics.svg)](https://badge.fury.io/py/pysysmetrics)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.7+](https://img.shields.io/badge/python-3.7+-blue.svg)](https://www.python.org/downloads/)

## Features

- **Real-time CPU Memory Monitoring** - Track memory usage of all running Python processes
- **GPU Memory Tracking** - Monitor NVIDIA GPU memory consumption per Python process
- **Detailed Process Information** - View PID, script name, CPU percentage, thread count, and memory usage
- **System Information Display** - Shows OS details, CPU cores, total RAM, and GPU specifications
- **Configurable Refresh Intervals** - Set custom monitoring intervals (in seconds)
- **Cross-Platform Support** - Works on Linux, macOS (CPU only), and Windows
- **Python API** - Use as a library in your own scripts for programmatic monitoring
- **Minimal Dependencies** - Only requires `psutil` and `pynvml` (for GPU)

## Use Cases

- **Machine Learning Development** - Monitor GPU memory during model training
- **Performance Profiling** - Track resource consumption of long-running scripts
- **Memory Leak Detection** - Identify memory growth patterns over time
- **Multi-Process Applications** - Monitor resource usage across parallel Python processes
- **DevOps & Monitoring** - Track Python service resource consumption
- **Educational** - Learn about system resource management and process monitoring

## Installation

```bash
pip install pysysmetrics
```

## Quick Start

### Monitor CPU Memory Usage

```bash
# Monitor CPU memory every 2 seconds
pysysmetrics --cpu 2
```

### Monitor GPU Memory Usage

```bash
# Monitor GPU memory every 5 seconds
pysysmetrics --gpu 5
```

## CLI Documentation

### Command Syntax

```bash
pysysmetrics [OPTIONS]
```

### Options

| Flag | Description | Example |
|------|-------------|---------|
| `--cpu N` | Monitor CPU memory usage every N seconds | `--cpu 2` |
| `--gpu N` | Monitor GPU memory usage every N seconds | `--gpu 5` |
| `-h, --help` | Show help message and exit | `-h` |

**Note:** If no arguments are provided, the help message will be displayed.

### Usage Examples

```bash
# Monitor CPU memory usage with 2-second refresh
pysysmetrics --cpu 2

# Monitor GPU memory usage with 5-second refresh
pysysmetrics --gpu 5

# Display help
pysysmetrics --help
```

## Output Examples

### CPU Monitoring Output

When you run `pysysmetrics --cpu 2`, you'll see output like:

```
----------------------------------------------------------------------
System Information:
Operating System : Darwin 25.0.0
----------------------------------------------------------------------
CPU Information:
CPU Model        : arm
CPU Cores        : 10
Total RAM        : 16.00 GB
----------------------------------------------------------------------

========================= Refresh @ 2025-01-15 14:23:45 =========================
PID       Script Name              CPU %     #Threads  Memory (MB)
--------------------------------------------------------------------------------
12345     train_model.py           85.2      8         2048.50
12346     data_processor.py        12.5      4         512.25
12347     api_server.py            5.3       2         256.10
======================================================================
```

**Output Fields:**
- **PID** - Process ID
- **Script Name** - Name of the Python script (extracted from command line)
- **CPU %** - Current CPU utilization percentage
- **#Threads** - Number of threads used by the process
- **Memory (MB)** - Resident Set Size (RSS) memory in megabytes

### GPU Monitoring Output

When you run `pysysmetrics --gpu 5`, you'll see output like:

```
----------------------------------------------------------------------
System Information:
Operating System : Linux 5.15.0
----------------------------------------------------------------------
GPU Information:
GPU 0 Model        : NVIDIA GeForce RTX 3090
GPU 0 Total Memory : 24.00 GB (24576 MB)
----------------------------------------------------------------------

========================= Refresh @ 2025-01-15 14:25:30 =========================
PID       Script Name                   GPU Mem (MB)   GPU Mem (GB)   GPU
--------------------------------------------------------------------------------
23456     train_model.py                8192.00        8.00           GPU 0
23457     inference.py                  2048.00        2.00           GPU 0
======================================================================
```

**Output Fields:**
- **PID** - Process ID
- **Script Name** - Name of the Python script
- **GPU Mem (MB)** - GPU memory used in megabytes
- **GPU Mem (GB)** - GPU memory used in gigabytes
- **GPU** - GPU device index

## Python API

You can use PySysMetrics programmatically in your Python scripts:

### CPU Monitoring

```python
from pysysmetrics.core import cpu

# Monitor CPU memory usage every 3 seconds
cpu.monitor_cpu_memory(interval=3)
```

This will continuously monitor and print CPU memory statistics every 3 seconds.

### GPU Monitoring

```python
from pysysmetrics.core import gpu

# Monitor GPU memory usage every 2 seconds
gpu.monitor_gpu_memory(interval=2)
```

This will continuously monitor and print GPU memory statistics every 2 seconds.

### Advanced Usage - Get Process Data

```python
from pysysmetrics.core.cpu import get_python_processes, print_python_processes_info

# Get list of all Python processes
processes = get_python_processes()

# Print formatted process information
print_python_processes_info(processes)
```

```python
from pysysmetrics.core.gpu import get_gpu_processes, print_gpu_processes_info

# Get list of Python processes using GPU
gpu_processes = get_gpu_processes()

# Print formatted GPU process information
print_gpu_processes_info(gpu_processes)
```

## Requirements

- **Python:** 3.7 or higher
- **CPU Monitoring:** Works on all platforms (Linux, macOS, Windows)
- **GPU Monitoring:**
  - NVIDIA GPU with CUDA support
  - NVIDIA drivers installed
  - Working `nvidia-smi` command
  - Not supported on macOS (pynvml limitation)

## Dependencies

PySysMetrics has minimal dependencies:

- **psutil** - Cross-platform process and system monitoring
- **pynvml** - Python bindings for NVIDIA Management Library (GPU monitoring)

These are automatically installed when you `pip install pysysmetrics`.

## Project Structure

```
pysysmetrics/
├── pysysmetrics/
│   ├── __init__.py
│   ├── cli/
│   │   ├── __init__.py
│   │   └── main.py              # CLI entry point and argument parsing
│   ├── core/
│   │   ├── __init__.py
│   │   ├── cpu.py               # CPU memory monitoring functions
│   │   └── gpu.py               # GPU memory monitoring functions
│   └── utils/
│       ├── __init__.py
│       ├── test_cpu.py          # CPU stress testing utility
│       └── test_gpu.py          # GPU stress testing utility
├── setup.py                     # Package configuration
├── pyproject.toml               # Build system configuration
├── README.md                    # This file
└── push.sh                      # Build and publish script
```

## Testing Utilities

PySysMetrics includes stress testing utilities to generate CPU and GPU load for testing the monitoring tools.

### CPU Stress Test

```bash
python -m pysysmetrics.utils.test_cpu
```

This script creates CPU and RAM load cycles:
- 5 seconds of CPU stress (busy-wait loop)
- 5 seconds of RAM allocation (~500MB)
- 5 seconds idle
- Repeats indefinitely

### GPU Stress Test

```bash
python -m pysysmetrics.utils.test_gpu
```

This script creates GPU load cycles using PyTorch:
- 5 seconds of GPU stress (matrix multiplication on CUDA)
- 5 seconds idle
- Repeats indefinitely

**Note:** GPU stress test requires PyTorch with CUDA support installed separately:
```bash
pip install torch
```

### Testing Workflow

1. **Terminal 1** - Run stress test:
   ```bash
   python -m pysysmetrics.utils.test_cpu
   ```

2. **Terminal 2** - Monitor the process:
   ```bash
   pysysmetrics --cpu 1
   ```

You should see the test script appear in the monitoring output with resource usage.

## Platform-Specific Notes

### Linux
- Full support for both CPU and GPU monitoring
- GPU monitoring requires NVIDIA drivers and CUDA

### macOS
- CPU monitoring fully supported
- GPU monitoring not supported (pynvml limitation)
- Uses system `psutil` library for process detection

### Windows
- CPU monitoring fully supported
- GPU monitoring requires NVIDIA drivers
- May need to run with administrator privileges for full process access

## Known Limitations

- **GPU Monitoring on macOS** - Not supported due to pynvml library limitations
- **Script Name Detection** - May show "N/A" for processes without `.py` files in command line
- **Permission Issues** - Some processes may not be accessible without elevated privileges
- **NVIDIA Only** - GPU monitoring only works with NVIDIA GPUs (uses NVML)
- **Monitoring Overhead** - Very minimal, but does consume some CPU for process enumeration

## Troubleshooting

### Issue: "No module named 'pynvml'"
**Solution:** Install pynvml: `pip install pynvml`

### Issue: "NVIDIA driver or GPU not detected"
**Solution:**
- Verify NVIDIA drivers are installed: `nvidia-smi`
- Ensure you have an NVIDIA GPU
- Check that CUDA is properly installed

### Issue: "GPU monitoring is not supported on macOS"
**Solution:** Use CPU monitoring instead (`--cpu`). GPU monitoring requires Linux or Windows with NVIDIA GPU.

### Issue: "No active Python processes found"
**Solution:**
- Run a Python script in another terminal
- Use the included test utilities to generate load
- Ensure processes are running and accessible

### Issue: "Permission denied" errors
**Solution:**
- Run with sudo (Linux/macOS): `sudo pysysmetrics --cpu 2`
- Run as Administrator (Windows)
- Note: Some system processes may still be inaccessible

## Contributing

Contributions are welcome! Here's how you can help:

1. **Fork the repository**
2. **Create a feature branch** (`git checkout -b feature/amazing-feature`)
3. **Make your changes**
4. **Test thoroughly** - Ensure CPU and GPU monitoring work correctly
5. **Commit your changes** (`git commit -m 'Add amazing feature'`)
6. **Push to the branch** (`git push origin feature/amazing-feature`)
7. **Open a Pull Request**

### Development Setup

```bash
# Clone the repository
git clone https://github.com/yourusername/pysysmetrics.git
cd pysysmetrics

# Install in development mode
pip install -e .

# Test the CLI
pysysmetrics --cpu 2
```

## License

This project is licensed under the MIT License - see the LICENSE file for details.

## Changelog

### Version 0.1.4
- Comprehensive README with complete documentation
- Enhanced package structure
- Improved testing utilities

### Version 0.1.2
- CPU core count and GPU percentage improvements
- Enhanced system information display

### Version 0.1.0
- Initial release
- CPU memory monitoring
- GPU memory monitoring
- CLI interface

## Acknowledgments

- Built with **psutil** for cross-platform process monitoring
- Uses **pynvml** for NVIDIA GPU management
- Inspired by tools like `nvidia-smi`, `htop`, and `top`
- Built for the Python and ML community

## Roadmap

- [ ] Support for AMD GPUs (ROCm)
- [ ] Historical data logging and visualization
- [ ] Export metrics to CSV/JSON
- [ ] Web dashboard for remote monitoring
- [ ] Docker container support
- [ ] Process filtering and search
- [ ] Alert thresholds for high memory usage
- [ ] Integration with Prometheus/Grafana

## Support

- **Author:** Aditya Thiyyagura
- **Email:** thiyyaguraadityareddy@gmail.com
- **Issues:** Report bugs and request features on GitHub

---

**Made with Python by Aditya Thiyyagura**