django-watchlog-apm 1.2.0
Django instrumentation for Watchlog APM with JSON OTLP export
License: MIT
Requires Python: >=3.7

# django_watchlog_apm

🔗 **Website**: https://watchlog.io

**django_watchlog_apm** is a lightweight APM (Application Performance Monitoring) integration for Django applications, built on OpenTelemetry. It automatically instruments your Django application to send traces to the Watchlog Agent.

## Features

- ✅ **Auto-instrumentation** for Django views, middleware, and HTTP requests
- ✅ **Database query tracing** (PostgreSQL via psycopg2)
- ✅ **JSON-over-HTTP exporter** (OTLP) compatible with Watchlog Agent
- ✅ **Environment detection** (local vs Kubernetes)
- ✅ **Configurable sampling** with error and slow span filtering
- ✅ **Non-blocking** - errors never crash your application
- ✅ **Zero configuration** - works out of the box

---

## Installation

### From PyPI

```bash
pip install django-watchlog-apm
```

**Note**: The package automatically includes PostgreSQL database query tracing support. If you're using PostgreSQL, database queries will be automatically traced without any additional installation.

---

## Quick Start

### Step 1: Initialize APM in `wsgi.py` or `asgi.py`

**Important**: You **must** call `instrument_django()` **before** Django loads its settings.

#### For WSGI (`wsgi.py`):

```python
import os
from django.core.wsgi import get_wsgi_application

# 1) Import and call instrumentation BEFORE Django setup
from django_watchlog_apm.instrument import instrument_django

instrument_django(
    service_name="my-django-app",  # Your service name
    otlp_endpoint="http://watchlog-agent:3774/apm",  # Watchlog agent endpoint
)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
application = get_wsgi_application()
```

#### For ASGI (`asgi.py`):

```python
import os
from django.core.asgi import get_asgi_application

# 1) Import and call instrumentation BEFORE Django setup
from django_watchlog_apm.instrument import instrument_django

instrument_django(
    service_name="my-django-app",
    otlp_endpoint="http://watchlog-agent:3774/apm",
)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
application = get_asgi_application()
```

### Step 2: Run Watchlog Agent

Make sure the Watchlog Agent is running and accessible. The agent should be listening on port `3774` with the `/apm` endpoint.

### Step 3: Test Your Application

Make requests to your Django application. Traces will be automatically sent to the Watchlog Agent.

---

## Configuration Options

### `instrument_django()` Parameters

| Parameter            | Type    | Default                     | Description                                                      |
| -------------------- | ------- | --------------------------- | ---------------------------------------------------------------- |
| `service_name`       | `str`   | **required**                | Name of your Django service (appears in Watchlog dashboard)      |
| `otlp_endpoint`      | `str`   | `http://localhost:3774/apm` | Base OTLP URL. Final URL will be `{endpoint}/{service_name}/v1/traces`. **Takes precedence over auto-detection if provided.** |
| `headers`            | `dict`  | `{}`                        | Additional HTTP headers for OTLP requests                         |
| `batch_max_size`     | `int`   | `200`                       | Maximum spans per batch export                                   |
| `batch_delay_ms`     | `int`   | `5000`                      | Delay (milliseconds) between batch exports                       |
| `sample_rate`        | `float` | `1.0`                       | Random sampling rate (0.0–1.0). **Note**: Internally capped at 0.3 for performance |
| `send_error_spans`   | `bool`  | `False`                     | If `True`, always export spans with error status                 |
| `error_tps`          | `int`   | `None`                      | Max error spans to export per second (`None` = unlimited)         |
| `slow_threshold_ms`  | `int`   | `0`                         | If >0, always export spans slower than this threshold (milliseconds) |
| `export_timeout`     | `float` | `10.0`                      | HTTP request timeout (seconds) for exporter POSTs                |

### Example: Advanced Configuration

```python
instrument_django(
    service_name="my-django-app",
    otlp_endpoint="http://watchlog-agent:3774/apm",
    sample_rate=1.0,              # Try to sample all requests (capped at 0.3 internally)
    send_error_spans=True,        # Always export error spans
    error_tps=10,                 # Max 10 error spans per second
    slow_threshold_ms=200,       # Always export spans >200ms
    batch_delay_ms=1000,          # Export batches every 1 second
    batch_max_size=50,           # Smaller batches for faster export
    export_timeout=5.0,          # 5 second timeout
)
```

---

## Docker Setup

### Connecting to Watchlog Agent in Docker

When running your Django app in Docker, you need to configure the correct agent endpoint using the `otlp_endpoint` parameter.

**Important**: The `otlp_endpoint` parameter takes precedence over auto-detection. Always specify it explicitly for Docker deployments.

```python
# wsgi.py or asgi.py
from django_watchlog_apm.instrument import instrument_django

instrument_django(
    service_name="my-django-app",
    otlp_endpoint="http://watchlog-agent:3774/apm",  # Use container name for Docker
)
```

### Docker Compose Example

```yaml
version: '3.8'

services:
  watchlog-agent:
    image: watchlog/agent:latest
    container_name: watchlog-agent
    ports:
      - "3774:3774"
    environment:
      - WATCHLOG_APIKEY=your-api-key
      - WATCHLOG_SERVER=your-server-address
    networks:
      - app-network

  django-app:
    build: .
    container_name: django-app
    ports:
      - "8000:8000"
    depends_on:
      - watchlog-agent
    networks:
      - app-network

networks:
  app-network:
    driver: bridge
```

### Docker Run Example

```bash
# 1. Create network
docker network create app-network

# 2. Run Watchlog Agent
docker run -d \
  --name watchlog-agent \
  --network app-network \
  -p 3774:3774 \
  -e WATCHLOG_APIKEY="your-api-key" \
  -e WATCHLOG_SERVER="your-server-address" \
  watchlog/agent:latest

# 3. Run Django app (make sure your code sets otlp_endpoint='http://watchlog-agent:3774/apm')
docker run -d \
  --name django-app \
  --network app-network \
  -p 8000:8000 \
  my-django-app
```

**Important Notes:**
- When using Docker, use the container name as the hostname (e.g., `watchlog-agent`)
- Both containers must be on the same Docker network
- The agent must be running before your app starts
- Always specify `otlp_endpoint` explicitly in your code for Docker deployments

---

## What Gets Traced?

### Automatically Traced:

- ✅ **HTTP Requests**: All incoming requests to Django views
- ✅ **Database Queries**: PostgreSQL queries (automatically enabled - no additional installation needed)
- ✅ **External HTTP Calls**: Outgoing requests made with the `requests` library
- ✅ **View Execution**: Django view function execution
- ✅ **Middleware**: Django middleware processing

### Trace Structure:

Each trace contains:
- **Root Span**: The HTTP request (e.g., `GET /api/users/`)
- **Child Spans**: Database queries, external HTTP calls, etc.

Example trace hierarchy:
```
GET /api/users/
  ├── SELECT * FROM users WHERE id = 1
  ├── GET https://external-api.com/data
  └── POST /api/users/1/update
```

---

## Manual Custom Spans

You can create custom spans using the OpenTelemetry API:

```python
from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def my_view(request):
    with tracer.start_as_current_span("custom.operation") as span:
        span.set_attribute("user.id", request.user.id)
        span.set_attribute("operation.type", "data_processing")
        
        # Your business logic here
        result = process_data()
        
        span.set_attribute("result.count", len(result))
        return JsonResponse({"result": result})
```

---

## Environment Detection

The package automatically detects the environment, but you can override it by providing the `otlp_endpoint` parameter:

**Priority order:**
1. **User-provided `otlp_endpoint`** (if provided, auto-detection is skipped)
2. **Auto-detection**:
   - **Local (non-Kubernetes)**: Uses `http://localhost:3774/apm`
   - **Kubernetes (in-cluster)**: Automatically switches to `http://watchlog-python-agent.monitoring.svc.cluster.local:3774/apm`

**Auto-detection checks (in order):**
1. Existence of `/var/run/secrets/kubernetes.io/serviceaccount/token`  
2. Presence of `kubepods` in `/proc/1/cgroup`  
3. DNS lookup of `kubernetes.default.svc.cluster.local`

**For Docker deployments**: Always specify `otlp_endpoint` explicitly to avoid auto-detection issues.  

---

## Error Handling

**Important**: This package is designed to never crash your application. All errors are silently handled:

- ✅ Export failures are swallowed (traces are lost, but app continues)
- ✅ Instrumentation failures are logged but don't prevent Django from starting
- ✅ PostgreSQL instrumentation is automatically included - no additional installation needed

If you want to see error logs, configure logging:

```python
import logging
logging.getLogger("django_watchlog_apm").setLevel(logging.INFO)
```

---

## Troubleshooting

### Traces Not Appearing in Watchlog

1. **Check Agent Connection**: Verify the Watchlog Agent is running and accessible
   ```bash
   curl http://watchlog-agent:3774/
   ```

2. **Check Network**: Ensure Django container can reach the agent
   ```bash
   docker exec django-app ping watchlog-agent
   ```

3. **Check Logs**: Look for export errors in Django logs
   ```bash
   docker logs django-app | grep "django_watchlog_apm"
   ```

4. **Verify Endpoint**: The final URL should be `http://watchlog-agent:3774/apm/{service_name}/v1/traces`

### Database Queries Not Traced

- **PostgreSQL instrumentation is automatically installed** with `django-watchlog-apm` - no additional packages needed
- Ensure you're using PostgreSQL with `psycopg2` or `psycopg2-binary` in your Django settings
- Check Django logs for: `[instrument_django] PostgreSQL/psycopg2 instrumentation enabled`
- If you see warnings about missing psycopg2 instrumentation, verify that `opentelemetry-instrumentation-psycopg2` was installed correctly

### High Memory Usage

- Reduce `batch_max_size` (default: 200)
- Increase `batch_delay_ms` to export less frequently
- Lower `sample_rate` (though it's capped at 0.3 internally)

---

## Sample Rate Limitation

**Important**: The `sample_rate` parameter is internally capped at **0.3** (30%) for performance reasons. Even if you set `sample_rate=1.0`, only approximately 30% of spans will be exported.

To export more spans:
- Set `slow_threshold_ms=0` to export all slow spans
- Set `send_error_spans=True` to always export error spans
- Use both in combination for maximum coverage

---

## License

MIT © Mohammadreza

Built for [Watchlog.io](https://watchlog.io)