geminiliveotel 0.2.9
Production observability for Gemini Live API on Vertex AI — latency, tokens, tool calls, audio compliance, and more.
Requires Python: >=3.11
Keywords: cloud-monitoring, gemini, live-api, native-audio, observability, opentelemetry, vertex-ai

# GeminiLiveOTel

Production observability for **Gemini Live API on Vertex AI** (native audio model). Zero-code-change telemetry sidecar that exports 28 metrics to Google Cloud Monitoring with an auto-generated dashboard.

## Install

```bash
pip install geminiliveotel
```

## Zero-change auto-instrumentation (recommended)

Add **one import** to your application entry point. No other code changes required.

```python
import geminiliveotel.auto  # ← patches google.genai.live on import
```

That's it. Every `client.aio.live.connect()` session is now instrumented. Metrics flow to Cloud Monitoring automatically.

```python
import geminiliveotel.auto

from google import genai

client = genai.Client(vertexai=True, project="my-project", location="us-central1")

# Your existing code — unchanged
async with client.aio.live.connect(model="gemini-live-2.5-flash-native-audio", config=config) as session:
    await session.send_realtime_input(audio=audio_blob)
    async for msg in session.receive():
        process(msg)
```

### Configuration

Set environment variables before your application starts:

| Variable | Default | Description |
|----------|---------|-------------|
| `GEMINI_OTEL_ENABLED` | `1` | Set to `0` or `false` to disable instrumentation |
| `GEMINI_OTEL_PROJECT` | — | GCP project ID for Cloud Monitoring export |
| `GEMINI_OTEL_REGION` | `us-central1` | GCP region |
| `GEMINI_OTEL_EXPORT` | `cloud_monitoring` | `cloud_monitoring`, `stdout`, or `none` |
| `GEMINI_OTEL_TARGET_MODEL` | `gemini-live-2.5-flash-native-audio` | Model to instrument |
| `GEMINI_OTEL_DASHBOARD` | `1` | Set to `0` to skip dashboard creation |
| `GEMINI_OTEL_QUOTA` | `1` | Set to `0` to skip quota fetch |
| `GEMINI_OTEL_FALLBACK_QUOTA` | `1000` | Fallback concurrency quota |
| `GEMINI_OTEL_FLUSH_INTERVAL` | `5` | Metric flush interval (seconds) |
| `GEMINI_OTEL_GROUP_BY_SESSION` | `0` | Set to `1` for session-grouped dashboard |
| `GEMINI_OTEL_CUMULATIVE_TOKENS` | `0` | Set to `1` if API reports cumulative token counts |
| `GEMINI_OTEL_RATE_WINDOW` | `60` | Sliding window size for QPS/throughput metrics (seconds) |

Example:

```bash
GEMINI_OTEL_PROJECT=my-project GEMINI_OTEL_REGION=us-central1 python myapp.py
```

### Explicit control

If you prefer explicit lifecycle management over import-time activation:

```python
from geminiliveotel.auto import instrument, uninstrument

instrument()      # apply patches
# ... run application ...
uninstrument()    # remove patches cleanly
```

## Explicit wrapper (alternative)

For fine-grained control over sidecar configuration, use the explicit wrapper pattern. This requires wrapping each session object.

```python
from geminiliveotel import bootstrap_vertex_live_observability, instrument_live_session
from google import genai

# 1. Bootstrap once at startup
obs = bootstrap_vertex_live_observability(
    project_id="my-project",
    region="us-central1",
    target_model="gemini-live-2.5-flash-native-audio",
)

# 2. Wrap each session
client = genai.Client(vertexai=True, project="my-project", location="us-central1")

async with client.aio.live.connect(model="gemini-live-2.5-flash-native-audio") as session:
    wrapped = instrument_live_session(
        session=session,
        sidecar=obs.sidecar,
        model_id="gemini-live-2.5-flash-native-audio",
        region="us-central1",
        automatic_vad=True,
    )

    # 3. Use wrapped session — same API, metrics captured automatically
    await wrapped.send_realtime_input(audio=audio_blob)
    async for msg in wrapped.receive():
        process(msg)
    await wrapped.close()
```

### Bootstrap options

```python
obs = bootstrap_vertex_live_observability(
    project_id="my-project",           # GCP project ID (required)
    region="us-central1",              # Vertex AI region (required)
    target_model="gemini-live-2.5-flash-native-audio",
    fallback_concurrency_quota=1000,   # Used if Quota API fetch fails
    auto_fetch_quota=True,             # Fetch real quota from Quota API
    auto_create_dashboard=True,        # Create Cloud Monitoring dashboard
    auto_export_metrics=True,          # Start background metric exporter
    metrics_flush_interval_sec=5,      # Flush interval
    group_by_session_id=False,         # False = aggregate across sessions
)
```

Returns a `BootstrapResult` with `sidecar`, `resolved_concurrency_quota`, `dashboard_name`, and `metrics_export_enabled`.

## Migration: explicit wrapper to auto-instrumentation

| Before | After |
|--------|-------|
| `bootstrap_vertex_live_observability(...)` | `import geminiliveotel.auto` + env vars |
| `instrument_live_session(session, sidecar, ...)` | Remove — sessions are auto-instrumented |
| `await wrapped.send_realtime_input(...)` | `await session.send_realtime_input(...)` |
| `async for msg in wrapped.receive():` | `async for msg in session.receive():` |
| `await wrapped.close()` | `await session.close()` |

Both patterns can coexist. Auto-instrumentation and explicit wrappers do not conflict.

## Architecture

```
┌──────────────────────────────────┐
│  Your application code           │
│  (no changes required)           │
└──────────────┬───────────────────┘
               │
  import geminiliveotel.auto
               │
               ▼
┌──────────────────────────────────┐
│  wrapt monkey-patches            │
│  google.genai.live.AsyncSession  │
│  .connect / .send_* / .receive   │
└──────────────┬───────────────────┘
               │ hooks
               ▼
┌──────────────────────────────────┐
│  LiveTelemetrySidecar            │
│  28 metrics (latency, tokens,    │
│  VAD, audio, errors, quota)      │
└──────────────┬───────────────────┘
               │ export
               ▼
┌──────────────────────────────────┐
│  Google Cloud Monitoring         │
│  + auto-generated dashboard      │
└──────────────────────────────────┘
```

The sidecar attaches to each session as a private attribute (`session._otel_sidecar`). With auto-instrumentation, a single shared sidecar instance is used across all sessions. With the explicit wrapper pattern, you control the sidecar lifecycle. The sidecar is fail-open — if anything goes wrong internally, your application continues unaffected.

## What you get

- Time-to-first-audio-byte (TTFAB) and turn latency percentiles
- Tool call roundtrip and model wait latency
- Audio chunk compliance (20–100 ms)
- Token consumption by modality
- QPS, concurrent sessions, quota utilization
- Error rates and success rate
- WebSocket connection latency
- Auto-generated Cloud Monitoring dashboard with session filtering

## Dashboard

The auto-generated dashboard includes:

| Section | Widgets |
|---------|---------|
| Key Metrics | Turns, Responses, Errors, Tool Calls (with sparklines) |
| Latency | TTFAB p50/p95/p99, Turn E2E p50/p95/p99, WS Connect |
| Tool & Audio | Tool Roundtrip, Model Wait, Audio Chunk Duration |
| Traffic & Health | Noncompliance Rate, Token Throughput, QPS, Sessions |
| Quota & Success | Quota Utilization %, Model Success Rate |
| Counters | Tokens, VAD, Interruptions, Errors, API Requests |
| Distribution | Modality Token Split |

Filter by `session_id`, `model_id`, or `region` in Cloud Monitoring's Group By dropdown.

## Metrics catalog (28 metrics)

### Histograms

| Metric | Unit | Description |
|--------|------|-------------|
| `geminilive_ttfab_ms` | ms | Time to first audio byte |
| `geminilive_turn_e2e_latency_ms` | ms | Turn end-to-end latency |
| `geminilive_tool_call_roundtrip_ms` | ms | Tool call roundtrip time |
| `geminilive_model_wait_after_tool_ms` | ms | Model wait after tool response |
| `geminilive_ws_connect_latency_ms` | ms | WebSocket connection time |
| `geminilive_input_audio_chunk_duration_ms` | ms | Audio chunk duration |
| `geminilive_input_audio_chunk_bytes` | bytes | Audio chunk size |

### Counters

| Metric | Description |
|--------|-------------|
| `geminilive_turns_total` | *(Deprecated)* Use `geminilive_turn_complete_total` instead |
| `geminilive_turn_complete_total` | Turn complete events |
| `geminilive_successful_model_responses_total` | Successful model responses |
| `geminilive_api_requests_total` | API requests sent |
| `geminilive_tool_calls_total` | Total tool calls requested |
| `geminilive_total_tokens_total` | Total tokens consumed |
| `geminilive_prompt_tokens_total` | Prompt tokens |
| `geminilive_response_tokens_total` | Response tokens |
| `geminilive_modality_tokens_total` | Tokens by media modality (audio/video/image; text excluded) |
| `geminilive_input_audio_chunk_noncompliant_total` | Audio chunks outside 20–100 ms |
| `geminilive_server_vad_events_total` | Server VAD events (automatic mode) |
| `geminilive_client_activity_signals_total` | Client activity signals (custom mode) |
| `geminilive_vad_interruptions_total` | VAD interruptions |
| `geminilive_api_errors_total` | API errors by error code |
| `geminilive_rate_limit_quota_errors_total` | Rate limit / quota errors |
| `geminilive_ws_errors_total` | WebSocket errors |

### Gauges

| Metric | Description |
|--------|-------------|
| `geminilive_request_qps` | Requests per second |
| `geminilive_token_throughput_tps` | Tokens per second |
| `geminilive_concurrent_sessions` | Currently active sessions |
| `geminilive_concurrency_quota_utilization_pct` | Quota utilization percentage |
| `geminilive_model_success_rate` | Model success rate (0–1) |

## API reference

### `geminiliveotel.auto`

| Function | Description |
|----------|-------------|
| `instrument() → bool` | Apply wrapt patches to `google.genai.live`. Returns `True` if applied, `False` if skipped or already active. |
| `uninstrument() → bool` | Remove all patches. Returns `True` if removed, `False` if nothing was patched. |
| `update_quota(new_quota: int) → bool` | Update concurrency quota on the shared sidecar. Returns `True` if applied, `False` if no sidecar exists. |

Auto-activation: `import geminiliveotel.auto` calls `instrument()` at import time. To prevent this, set `GEMINI_OTEL_ENABLED=0` before importing.

### `geminiliveotel.integration`

| Function | Description |
|----------|-------------|
| `instrument_live_session(session, sidecar, ...) → InstrumentedLiveSession` | Wrap a session for explicit instrumentation. |

### `geminiliveotel.bootstrap`

| Function | Description |
|----------|-------------|
| `bootstrap_vertex_live_observability(...) → BootstrapResult` | One-shot setup: sidecar + exporter + dashboard. |

## Authentication

Uses Google Cloud Application Default Credentials (ADC):

```bash
# Service account
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"

# User credentials
gcloud auth application-default login
```

## Development

```bash
uv sync                          # Install dependencies
uv run pytest tests/ -v          # Run all tests
uv build                         # Build package
```

## License

Apache 2.0