binary-protocol 0.2.2
A lightweight binary protocol implementation
Requires Python: >=3.8

# Custom Binary Protocol API

## Requirements

- Python 3.8 or higher
- asyncio
- pytest
- matplotlib & seaborn
- psutil

## Overview

This project implements a lightweight, configurable binary protocol for client-server communication in Python. Unlike traditional REST APIs, this implementation uses a custom binary protocol for data transfer, resulting in lower latency and reduced network overhead.

### Key Features

- **Structured Message Format**: Define message structures with typed fields
- **Efficient Data Transfer**: Custom binary protocol minimizes overhead compared to HTTP/REST
- **Type-safe Field Handling**: Automatic serialization of primitive types
- **Asynchronous Communication**: Built on asyncio for high-performance I/O operations
- **Zero Dependencies**: Pure Python implementation with no external requirements
- **Bi-directional Communication**: Supports full-duplex communication between client and server
- **Fixed and Variable Length Fields**: Support for both fixed and variable-length data
- **Optimized Performance**: Fast-path for fixed-length messages

## Quick Examples

### Message Type Definition

```python
from enum import IntEnum
from binary_protocol.protocol.message_structure import FieldDefinition, MessageStructure

# Define message types
class MessageType(IntEnum):
    SENSOR_DATA = 1
    STATUS = 2
    BENCHMARK = 3
    FIXED_BENCHMARK = 4

# Define fixed-length structure for high-speed sensor data
sensor_data_structure = MessageStructure([
    FieldDefinition("sensor1", float, format_string=">f"),  # 4-byte float
    FieldDefinition("sensor2", float, format_string=">f"),
    FieldDefinition("sensor3", float, format_string=">f"),
    FieldDefinition("sensor4", float, format_string=">f")
])

# Variable-length structure example
status_structure = MessageStructure([
    FieldDefinition("status_code", int, format_string=">I"),
    FieldDefinition("message", str)  # Variable length
])

# Fixed-length benchmark structure
fixed_benchmark_structure = MessageStructure([
    FieldDefinition("value", int, format_string=">I")  # 4-byte integer
])
```

### Protocol Configuration

```python
from binary_protocol import ProtocolConfig

config = ProtocolConfig(
    message_type_enum=MessageType,
    message_structures={
        MessageType.SENSOR_DATA: sensor_data_structure,
        MessageType.STATUS: status_structure,
        MessageType.BENCHMARK: benchmark_structure,
        MessageType.FIXED_BENCHMARK: fixed_benchmark_structure
    },
    header_format=">H",  # 2-byte header for message type
    max_payload_size=10 * 1024 * 1024  # 10MB
)
```

## Benchmarking Features

### Available Benchmark Types

1. **Fixed Size Benchmark**
   - Optimized for consistent message sizes
   - Validates message integrity
   - Uses predefined message templates

2. **Variable Size Benchmark**
   - Tests multiple message sizes
   - Measures scaling characteristics
   - Configurable size ranges

3. **Realistic Benchmark**
   - Multi-client simulation
   - Random message sizes
   - Configurable client behavior

### Running Benchmarks

```bash
python examples/basic_client.py --benchmark [options]
```

Options:
- `--all`: Run all benchmark types
- `--fixed`: Run fixed-size benchmarks
- `--variable`: Run variable-size benchmarks
- `--realistic`: Run realistic benchmarks

### Output Formats

#### Console Output
- Detailed benchmark summary tables
- Resource utilization metrics
- Statistical analysis
- Performance comparisons

#### Generated Files
- JSON result files
- Performance plots (PNG)
- Comparison reports
- Resource usage graphs

## Protocol Configuration Options

| Option | Description | Default |
|--------|-------------|---------|
| message_type_enum | Enum class for message types | Required |
| message_structures | Dict of structures per type | Required |
| header_format | Struct format for message header | ">H" |
| max_payload_size | Maximum allowed payload size | 1MB |
| connect_timeout | Connection timeout in seconds | 30.0 |
| read_timeout | Read operation timeout | 30.0 |

## Field Types and Performance

| Field Type | Format | Performance Impact | Batch Support |
|------------|--------|-------------------|---------------|
| int/float (fixed) | ">f", ">i", etc. | Fastest | Up to 100k msgs/batch |
| fixed-length str/bytes | fixed_length=N | Fast | Up to 50k msgs/batch |
| variable-length str/bytes | None | Additional overhead | Dynamic batch size |

## Security Considerations

1. **Encryption**: Consider adding TLS for secure communication
2. **Authentication**: Implement a secure authentication mechanism
3. **Input Validation**: Validate field contents
4. **Rate Limiting**: Protect against DoS attacks
5. **Message Size Limits**: Configure appropriate max_payload_size

## Performance Optimizations

When `enable_performance_improvements` is True (default):
- Uses message pooling to reduce GC pressure
- Implements zero-copy operations where possible
- Optimizes fixed-length message handling
- Reduces memory allocations

Example configuration with performance improvements:
```python
config = ProtocolConfig(
    message_type_enum=MessageType,
    message_structures={
        MessageType.SENSOR_DATA: sensor_data_structure,
        MessageType.STATUS: status_structure
    },
    enable_performance_improvements=True  # Enable optimizations
)
```

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.