dogukan-mlops-metrics 1.0.0
A simple MLOps metrics library for batch prediction monitoring
License: MIT
Requires Python: >=3.7
Keywords: mlops, metrics, prometheus, monitoring, machine learning

# MLOps Metrics

A simple Python library for monitoring MLOps batch prediction processes using Prometheus metrics.

## Features

- Track batch processing success/failure rates
- Monitor record processing counts
- Measure batch execution times
- Track current processing offsets
- Push metrics to Prometheus Push Gateway
- Support for multiple model versions and environments
- Configurable namespace and gateway URL

## Installation

```bash
pip install mlops-metrics
```

Or install from source:

```bash
git clone <repository-url>
cd mlops-metrics
pip install -e .
```

## Quick Start

```python
from mlops_metrics import MLOpsMetrics

# Initialize metrics with default settings
metrics = MLOpsMetrics(
    model_version="v1.0.0",
    environment="production"
)

# Or with custom namespace and gateway URL
metrics = MLOpsMetrics(
    model_version="v1.0.0",
    environment="production",
    namespace="my-namespace",
    gateway_url="my-gateway:9091"
)

# Record successful batch
metrics.record_batch_success(
    execution_time=2.5,
    record_count=1000,
    offset=5000
)

# Record failed batch
metrics.record_batch_failure()

# Push metrics to Prometheus Push Gateway
metrics.push_metrics()
```

## Quick Start

For a quick test of the library:

```bash
# Make the script executable and run it
chmod +x quickstart.sh
./quickstart.sh
```

This will:
1. Create a virtual environment
2. Install the library and dependencies
3. Run the dummy metrics sender example
4. Show you the next steps

## Configuration

The library supports the following configuration parameters:

- `model_version`: Version of your ML model (default: "v1.0.0")
- `environment`: Deployment environment (dev, staging, production) (default: "production")
- `namespace`: Kubernetes namespace for Push Gateway (default: "prometheus-pushgateway")
- `gateway_url`: Custom Push Gateway URL. If None, automatically constructed as `prometheus-pushgateway.{namespace}.svc.cluster.local:9091`

## Metrics

The library exposes the following Prometheus metrics:

- `mlops_batch_processed_total`: Total number of processed batches (success/failure)
- `mlops_records_processed_total`: Total number of processed records
- `mlops_batch_execution_seconds`: Last batch execution time
- `mlops_current_offset`: Current processing offset

## Example Usage

### Basic Example

```python
from mlops_metrics import MLOpsMetrics

# Initialize metrics with default settings
metrics = MLOpsMetrics(
    model_version="v1.0.0",
    environment="production"
)

# Or with custom namespace and gateway URL
metrics = MLOpsMetrics(
    model_version="v1.0.0",
    environment="production",
    namespace="my-namespace",
    gateway_url="my-gateway:9091"
)

# Record successful batch
metrics.record_batch_success(
    execution_time=2.5,
    record_count=1000,
    offset=5000
)

# Record failed batch
metrics.record_batch_failure()

# Push metrics to Prometheus Push Gateway
metrics.push_metrics()
```

### Dummy Metrics Sender Example

A complete example that sends dummy metrics to demonstrate the library:

```bash
# Run the dummy metrics sender
python examples/dummy_metrics_sender.py
```

This example will:
- Initialize MLOps metrics
- Simulate 5 batch processing cycles
- Generate random success/failure events
- Try to push metrics to Push Gateway (localhost:9091)
- Display helpful information about setting up Prometheus

To use with a real Push Gateway:

```bash
# Start Prometheus Push Gateway
docker run -d -p 9091:9091 prom/pushgateway

# Run the example
python examples/dummy_metrics_sender.py
```

## License

MIT License

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.