netrun-logging 2.0.0
Unified structured logging service with Azure App Insights integration and namespace packaging
License: MIT
Requires Python: >=3.9
Keywords: azure, correlation-id, fastapi, logging, namespace, structured-logging

# Service #61: Netrun Unified Logging

**Version**: 2.0.0
**Status**: Production Ready - PyPI Package Available
**Last Updated**: December 18, 2025
**PyPI**: `netrun-logging`

---

## MIGRATION NOTICE: v2.0.0 Namespace Packaging

**Version 2.0.0 introduces namespace packaging.** Import paths have changed:

```python
# ❌ OLD (deprecated, will be removed in v3.0.0):
from netrun_logging import configure_logging, get_logger
from netrun_logging.middleware import add_logging_middleware

# ✅ NEW (required for v2.0.0+):
from netrun.logging import configure_logging, get_logger
from netrun.logging.middleware import add_logging_middleware
```

**Backwards compatibility shim included** - old imports will work with a deprecation warning until v3.0.0.

See [MIGRATION_v2.0.0.md](./MIGRATION_v2.0.0.md) for complete migration guide.

---

## Overview

Unified structured logging service for the Netrun Service Library. Provides high-performance structured logging with structlog backend, correlation ID tracking, context enrichment, and optional Azure App Insights integration.

**Key Features**:
- **NEW v1.1.0**: Structlog backend for 2-3x faster logging performance
- **NEW v1.1.0**: Async logging support (logger.ainfo, logger.aerror, etc.)
- **NEW v1.1.0**: Automatic sensitive field sanitization (passwords, API keys, tokens)
- **NEW v1.1.0**: OpenTelemetry trace injection for distributed tracing
- **NEW v1.1.0**: Enhanced context management with bind_context()
- Structured JSON logging with ISO 8601 timestamps
- Thread-safe correlation ID tracking for distributed tracing
- Application and request context enrichment
- FastAPI middleware integration
- Azure App Insights telemetry support (optional)
- Zero-configuration defaults with extensive customization

---

## Quick Start

### Installation

**From PyPI** (recommended):
```bash
pip install netrun-logging
```

**From Source** (development):
```bash
git clone https://github.com/netrun-services/netrun-logging
cd netrun-logging
pip install -e ".[dev]"
```

### Basic Usage

```python
from netrun.logging import configure_logging, get_logger

# Configure logging (call once at application startup)
configure_logging(
    app_name="my-service",
    environment="production",
    log_level="INFO"
)

# Get logger instance
logger = get_logger(__name__)

# Log messages
logger.info("Application started", extra={"version": "1.0.0"})
logger.error("Operation failed", extra={"error_code": "E001"})
```

**Output** (JSON):
```json
{
  "timestamp": "2025-11-24T18:45:32.123456+00:00",
  "level": "INFO",
  "message": "Application started",
  "logger": "my_service",
  "extra": {
    "version": "1.0.0"
  }
}
```

---

## What's New in v2.0.0

### Namespace Packaging Migration

v2.0.0 migrates to namespace packaging (`netrun.logging`) for better ecosystem integration:

```python
from netrun.logging import configure_logging, get_logger

configure_logging(app_name="my-service", environment="production")
logger = get_logger(__name__)

# Structlog's key-value API (recommended)
logger.info("user_login", user_id=12345, ip="192.168.1.1", duration=1.23)

# Traditional message format still works
logger.info("User 12345 logged in from 192.168.1.1")
```

**Performance Benefits**:
- 2-3x faster log processing
- Lower memory overhead
- Async logging support

### Async Logging Support

All logging methods now have async counterparts:

```python
import asyncio
from netrun.logging import configure_logging, get_logger

configure_logging(app_name="async-service")
logger = get_logger(__name__)

async def process_request():
    await logger.ainfo("async_request_started", request_id=123)

    result = await perform_operation()

    await logger.ainfo("async_request_completed", request_id=123, result=result)

asyncio.run(process_request())
```

**Available Async Methods**:
- `logger.ainfo()` - Async info logging
- `logger.aerror()` - Async error logging
- `logger.awarning()` - Async warning logging
- `logger.adebug()` - Async debug logging
- `logger.acritical()` - Async critical logging

### Automatic Sensitive Field Sanitization

Sensitive fields are automatically redacted to prevent credential leaks:

```python
logger.info("api_request",
    user="alice",
    password="secret123",  # Automatically redacted
    api_key="sk-abc123",   # Automatically redacted
    token="bearer xyz"     # Automatically redacted
)

# Output:
# {"event": "api_request", "user": "alice", "password": "[REDACTED]", "api_key": "[REDACTED]", "token": "[REDACTED]"}
```

**Redacted Fields** (case-insensitive):
- `password`, `PASSWORD`, `Password`
- `api_key`, `apikey`, `API_KEY`
- `secret`, `SECRET_VALUE`
- `token`, `access_token`, `refresh_token`
- `authorization`, `auth`
- `credential`, `private_key`

### OpenTelemetry Trace Injection

When running with OpenTelemetry instrumentation, trace context is automatically added:

```python
from opentelemetry import trace
from netrun.logging import configure_logging, get_logger

configure_logging(app_name="traced-service")
logger = get_logger(__name__)

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("process_request"):
    logger.info("processing_request")
    # Automatically includes trace_id and span_id for correlation

# Output:
# {"event": "processing_request", "trace_id": "0af7651916cd43dd8448eb211c80319c", "span_id": "b7ad6b7169203331"}
```

### Enhanced Context Management

New `bind_context()` function for adding persistent context to all logs:

```python
from netrun.logging import configure_logging, get_logger, bind_context, clear_context

configure_logging(app_name="context-service")
logger = get_logger(__name__)

# Bind context for all subsequent logs
bind_context(user_id="user-12345", tenant_id="acme-corp", session_id="sess-789")

logger.info("user_action")  # Includes user_id, tenant_id, session_id
logger.info("another_action")  # Still includes context

# Clear context when done
clear_context()
logger.info("logged_out")  # No context fields
```

---

## Correlation ID Tracking

Track requests across distributed services:

```python
from netrun.logging import configure_logging, get_logger, correlation_id_context

configure_logging(app_name="api-gateway")
logger = get_logger(__name__)

# Automatic correlation ID generation
with correlation_id_context() as correlation_id:
    logger.info("Request received", extra={"correlation_id": correlation_id})
    process_request(correlation_id)
    logger.info("Request completed", extra={"correlation_id": correlation_id})
```

**Output**:
```json
{"timestamp": "2025-11-24T18:45:32.123456+00:00", "level": "INFO", "message": "Request received", "logger": "api_gateway", "correlation_id": "2da84060-098c-44bb-90a4-4b5ab8836c4d"}
{"timestamp": "2025-11-24T18:45:32.456789+00:00", "level": "INFO", "message": "Request completed", "logger": "api_gateway", "correlation_id": "2da84060-098c-44bb-90a4-4b5ab8836c4d"}
```

---

## Context Enrichment

Add application metadata to all logs:

```python
from netrun.logging import configure_logging, get_logger, set_context

configure_logging(app_name="user-service")
logger = get_logger(__name__)

# Set context for current request
set_context(
    app_name="user-service",
    environment="production",
    version="2.3.1",
    user_id="user-12345",
    tenant_id="tenant-abc"
)

logger.info("User action logged")
```

---

## FastAPI Integration

```python
from fastapi import FastAPI
from netrun.logging import configure_logging
from netrun.logging.middleware.fastapi import LoggingMiddleware

app = FastAPI()

# Configure logging
configure_logging(
    app_name="my-api",
    environment="production"
)

# Add middleware
app.add_middleware(LoggingMiddleware)

@app.get("/health")
def health_check():
    return {"status": "healthy"}
```

**Automatic Request Logging**:
- Request ID generation
- Request method, path, query parameters
- Response status code and duration
- Exception tracking

---

## Azure App Insights Integration

```python
from netrun.logging import configure_logging

configure_logging(
    app_name="production-api",
    environment="production",
    azure_insights_connection_string="InstrumentationKey=your-key-here"
)
```

**Telemetry Sent to Azure**:
- Structured logs with custom properties
- Request telemetry (duration, status)
- Exception telemetry with stack traces
- Custom events and metrics

---

## Configuration Options

### `configure_logging()`

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `app_name` | `str` | `"app"` | Application name for log context |
| `environment` | `str` | `$ENVIRONMENT` or `"development"` | Environment name (dev, staging, prod) |
| `log_level` | `str` | `"INFO"` | Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL) |
| `enable_json` | `bool` | `True` | Use JSON formatter |
| `enable_correlation_id` | `bool` | `True` | Enable correlation ID tracking |
| `azure_insights_connection_string` | `str` | `None` | Azure App Insights connection string |

---

## Architecture

### Package Structure

```
netrun_logging/
├── __init__.py              # Package entry point
├── logger.py                # Core logger configuration
├── correlation.py           # Correlation ID management
├── context.py               # Log context management
├── formatters/
│   ├── __init__.py
│   └── json_formatter.py    # JSON log formatter
├── middleware/
│   ├── __init__.py
│   └── fastapi.py           # FastAPI middleware
├── integrations/
│   ├── __init__.py
│   └── azure_insights.py    # Azure App Insights integration
└── integration_templates/   # Project-specific integration templates
    ├── README.md            # Integration guide
    ├── intirkon.py          # Multi-tenant Azure BI platform
    ├── netrun_crm.py        # CRM with lead scoring
    ├── intirkast.py         # Content creator SaaS
    ├── wilbur.py            # Charlotte AI bridge
    ├── securevault.py       # Secrets management with audit logging
    ├── dungeonmaster.py     # Game server with combat logging
    ├── ghostgrid.py         # FSO network simulation
    ├── intirfix.py          # Service dispatch platform
    ├── netrun_site.py       # Marketing website API
    ├── eiscore.py           # Unreal Engine Python scripting
    └── service_library.py   # Documentation scripts
```

### Core Components

**JsonFormatter** (`formatters/json_formatter.py`):
- Custom `logging.Formatter` for JSON output
- ISO 8601 timestamp formatting
- Exception serialization with traceback

**Correlation ID** (`correlation.py`):
- Thread-safe `contextvars.ContextVar` storage
- UUID4 generation
- Context manager for request scoping

**Log Context** (`context.py`):
- Application metadata (app_name, environment, version)
- Request metadata (user_id, tenant_id)
- Custom fields via `extra` dict

---

## Code Reuse Sources

**60% Code Reuse** from Netrun portfolio:

1. **GhostGrid AuditLogger** (40%):
   - JSON formatter design
   - Structured metadata patterns
   - Exception serialization

2. **Intirkast Correlation ID** (20%):
   - `contextvars` implementation
   - FastAPI middleware pattern
   - Thread-safe UUID generation

3. **New Development** (40%):
   - Azure App Insights integration
   - Unified API design
   - Context enrichment system

---

## Development Status

### Week 1: Core API (November 24, 2025) ✅

- [x] Package structure and configuration
- [x] Core logger implementation
- [x] JSON formatter
- [x] Correlation ID management
- [x] Log context management
- [x] Basic import validation

### Week 1: Testing & Documentation ✅

- [x] Unit tests for core modules (77% coverage, 38 tests passing) ✅ December 3, 2025
- [x] Integration tests for FastAPI middleware ✅ December 3, 2025
- [x] Azure App Insights integration implementation ✅ December 3, 2025
- [x] API documentation (November 24, 2025)
- [x] Usage examples and quickstart guide (November 24, 2025)

### Week 2: Packaging & Deployment ✅

- [x] PyPI packaging configuration ✅ December 3, 2025
- [x] Distribution built and validated (v1.1.0) ✅ December 3, 2025
- [x] Integration templates for 11 projects (November 24, 2025)
- [x] Test suite validation (38/38 tests passing) ✅ December 3, 2025
- [ ] PyPI publication (pending PyPI account setup)
- [ ] Performance benchmarks
- [ ] Production deployment checklist

---

## Testing

**Test Suite**: 38 tests | **Coverage**: 77% | **Status**: All Passing ✅

```bash
# Run all tests with coverage
pytest tests/ -v --cov=netrun_logging --cov-report=html

# Results (December 3, 2025):
# ============================= 38 passed in 2.22s ==============================
# Coverage: 77% (260 statements, 59 missing)

# Run type checking
mypy netrun_logging/

# Run linting
ruff check netrun_logging/
```

**Test Coverage Breakdown** (December 3, 2025):
- `correlation.py`: 91% (23 statements, 2 missing)
- `formatters/json_formatter.py`: 95% (39 statements, 2 missing)
- `middleware/fastapi.py`: 90% (42 statements, 4 missing)
- `logger.py`: 87% (38 statements, 5 missing)
- `processors.py`: 79% (43 statements, 9 missing)
- `context.py`: 93% (30 statements, 2 missing)
- `integrations/azure_insights.py`: 0% (33 statements, untested - optional dependency)

**Build Artifacts** (December 3, 2025):
- `netrun_logging-1.1.0-py3-none-any.whl`: 19 KB ✅ Validated
- `netrun_logging-1.1.0.tar.gz`: 23 KB ✅ Validated

---

## Documentation

Complete MkDocs documentation is available:

```bash
# View documentation locally
mkdocs serve

# Visit http://localhost:8000 in browser
```

### Documentation Structure

- **Home** (`docs/index.md`) - Overview and quick links
- **Getting Started** (`docs/getting-started.md`) - Installation and basic usage
- **Configuration** (`docs/configuration.md`) - All configuration options
- **API Reference**:
  - `configure_logging()` - Logging configuration
  - `get_logger()` - Logger instantiation
  - Correlation ID management
  - Log context management
  - JSON formatter details
  - FastAPI middleware integration
- **Examples**:
  - Basic usage patterns
  - FastAPI integration guide
  - Azure App Insights setup
- **Changelog** - Version history and roadmap

### Building Documentation

```bash
# Install MkDocs and Material theme
pip install mkdocs mkdocs-material

# Build static site
mkdocs build

# Output: ./site/ directory ready for deployment
```

### Documentation Features

- Material design theme with dark mode support
- Full-text search across all pages
- Code syntax highlighting for Python
- Markdown tables and admonitions
- Navigation tabs and section grouping
- Mobile-responsive layout

---

## Integration Roadmap

**Target Projects** (11 total):
1. Intirkast (Podcast platform)
2. Intirkon (Conference platform)
3. GhostGrid Optical Network
4. Meridian (Issuu alternative)
5. SecureVault
6. Charlotte (Agent orchestration)
7. Wilbur (Knowledge agent)
8. NetrunnewSite
9. Netrun CRM
10. EISCore 5.6 (Unreal)
11. DungeonMaster

**Integration Strategy**:
- Replace project-specific logging with `netrun-logging`
- Standardize correlation ID patterns
- Centralize Azure App Insights telemetry
- Reduce duplicate logging code (2,400+ LOC estimated)

---

## ROI Analysis

**Annual Cost Savings**: $33,440
- Reduced logging maintenance: 220 hours/year ($22,000)
- Faster debugging with correlation IDs: 80 hours/year ($8,000)
- Centralized monitoring setup: 34 hours/year ($3,440)

**Investment**: $12,500 (11.5-hour recreation sprint)
**ROI**: 267%
**Payback Period**: 2.7 months

---

## Support

**Documentation**: See `WEEK_1_RECREATION_ROADMAP.md` for detailed implementation plan
**Service Catalog**: Service #61 in Netrun Services Master Sales Sheet
**Integration Guide**: See `IMPLEMENTATION_BACKLOG.md` for Phase 1 rollout plan

**Contact**:
- Daniel Garza, Netrun Systems
- Email: daniel@netrunsystems.com
- SDLC Compliance: v2.2

---

## License

Proprietary - Netrun Systems (C Corporation)