npm package: @socialblobs/core
Description: Social-Blobs protocol reference implementation
Author: Wonderland
Latest version: 0.3.0

# Social-Blobs Protocol Reference Implementation

[![CI](https://github.com/wonderland/socialblobs-core/actions/workflows/ci.yml/badge.svg)](https://github.com/wonderland/socialblobs-core/actions/workflows/ci.yml)
[![npm version](https://img.shields.io/npm/v/@socialblobs/core)](https://www.npmjs.com/package/@socialblobs/core)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Node Version](https://img.shields.io/badge/node-%3E%3D20.0.0-brightgreen)](https://nodejs.org/)

Reference implementation of the Social-Blobs protocol - a minimal social media protocol built on
Ethereum's EIP-4844 blob data.

## Features

- **📝 Message Encoding/Decoding**: Binary message format with UTF-8 content support
- **📦 Batch Assembly**: Efficient multi-message batching with author tables
- **🗜️ Compression**: Zstd compression with trained dictionary (5-10x ratios)
- **✍️ Signatures**: BLS12-381 aggregation and ECDSA support
- **🔗 Aggregator Client**: REST API client for message submission
- **🛠️ CLI Tools**: Command-line utilities for encoding, decoding, and validation
- **✅ Test Vectors**: Comprehensive test vectors for implementation validation

## Installation

```bash
npm install @socialblobs/core
```

### Requirements

- Node.js >= 20.0.0
- npm or yarn

## Quick Start

### Encoding a Message

```typescript
import { encodeMessage, computeMessageHash } from '@socialblobs/core';

const message = {
  author: '0x1234567890abcdef1234567890abcdef12345678',
  timestamp: Math.floor(Date.now() / 1000),
  nonce: 0,
  content: 'Hello, Social-Blobs!',
  signatureType: 'bls' as const,
};

// Encode to binary
const encoded = encodeMessage(message);
console.log(`Encoded: ${encoded.length} bytes`);

// Compute message hash
const hash = computeMessageHash(encoded);
console.log(`Hash: ${hash}`);
```

### Decoding a Message

```typescript
import { decodeMessage, bytesToHex } from '@socialblobs/core';

// Decode from binary
const decoded = decodeMessage(encoded);
console.log(`Author: ${decoded.author}`);
console.log(`Content: ${decoded.content}`);
console.log(`Timestamp: ${decoded.timestamp}`);
```

### Creating a Batch

```typescript
import { encodeBatch, decodeBatch } from '@socialblobs/core';

const messages = [
  {
    author: '0x1111111111111111111111111111111111111111',
    timestamp: 1706400000,
    nonce: 0,
    content: 'First message',
    signatureType: 'bls' as const,
  },
  {
    author: '0x2222222222222222222222222222222222222222',
    timestamp: 1706400100,
    nonce: 0,
    content: 'Second message',
    signatureType: 'bls' as const,
  },
];

// Encode batch with compression
const batch = await encodeBatch({
  messages,
  baseTimestamp: 1706400000,
  compress: true,
  dictionaryRef: '0x0000000000000000000000000000000000000000000000000000000000000000',
});

console.log(`Batch size: ${batch.blob.length} bytes`);
console.log(`Messages: ${batch.metadata.messageCount}`);
console.log(`Authors: ${batch.metadata.authorCount}`);

// Decode batch
const decoded = await decodeBatch(batch.blob);
console.log(`Decoded ${decoded.messages.length} messages`);
```

### BLS Signatures

```typescript
import {
  generateBLSPrivateKey,
  deriveBLSPublicKey,
  signBLS,
  verifyBLS,
  aggregateBLS,
  verifyAggregateBLS,
} from '@socialblobs/core';

// Generate key pair
const privateKey = generateBLSPrivateKey();
const publicKey = await deriveBLSPublicKey(privateKey);

// Sign message hash
const messageHash = computeMessageHash(encoded);
const signature = await signBLS(privateKey, messageHash);

// Verify signature
const valid = await verifyBLS(publicKey, messageHash, signature);
console.log(`Signature valid: ${valid}`);

// Aggregate multiple signatures
const signatures = [signature1, signature2, signature3];
const aggregateSignature = aggregateBLS(signatures);

// Verify aggregate
const publicKeys = [pubKey1, pubKey2, pubKey3];
const messageHashes = [hash1, hash2, hash3];
const aggregateValid = await verifyAggregateBLS(publicKeys, messageHashes, aggregateSignature);
```

### Aggregator Client

```typescript
import { AggregatorClient } from '@socialblobs/core';

// Connect to aggregator
const client = new AggregatorClient({
  baseUrl: 'https://aggregator.socialblobs.eth',
  timeout: 5000,
});

// Check health
const health = await client.health();
console.log(`Status: ${health.status}`);

// Submit message
const result = await client.submit({
  message: encoded,
  signature: signature,
});
console.log(`Message ID: ${result.messageId}`);

// Check status
const status = await client.status(result.messageId);
console.log(`Status: ${status.status}`);
```

## CLI Tools

The package includes a command-line tool for working with Social-Blobs messages and batches.

### Message Commands

```bash
# Encode a message
socialblobs message encode "Hello, world!" \
  --author 0x1111111111111111111111111111111111111111 \
  --timestamp 1706400000

# Decode a message
socialblobs message decode 0x534f424d... --json

# Compute message hash
socialblobs message hash 0x534f424d...

# Compute signing hash
socialblobs message hash 0x534f424d... --signing-hash
```

### Batch Commands

```bash
# Create a batch from JSON file
socialblobs batch create --input messages.json --compress

# Decode a batch
socialblobs batch decode 0x534f4231... --json

# Show batch statistics
socialblobs batch decode 0x534f4231... --stats
```

### Validation

```bash
# Validate message format
socialblobs validate message 0x534f424d...

# Validate batch format
socialblobs validate batch 0x534f4231...

# Run test vectors
socialblobs validate vectors tests/vectors/messages.json
```

## API Reference

### Core Modules

- **`message`** - Message encoding, decoding, and hashing
- **`batch`** - Batch assembly, encoding, and decoding
- **`compression`** - Zstd compression with dictionary support
- **`signatures`** - BLS12-381 and ECDSA signature operations
- **`aggregator-client`** - REST API client for aggregators

### Types

```typescript
interface Message {
  author: Address; // 20-byte Ethereum address
  timestamp: number; // Unix timestamp (seconds)
  nonce: number; // Nonce (0-65535)
  content: string; // UTF-8 content (max 280 chars, 1120 bytes)
  signatureType: SignatureType; // 'bls' | 'ecdsa'
  replyTo?: Bytes32; // Optional reply reference
}

interface Batch {
  baseTimestamp: number; // Base timestamp for delta encoding
  messages: Message[]; // Messages to batch
  dictionaryRef: Bytes32; // Compression dictionary reference
  compress?: boolean; // Enable compression (default: true)
}

interface EncodedBatch {
  blob: Uint8Array; // Encoded batch data
  metadata: {
    messageCount: number;
    authorCount: number;
    compressedSize?: number;
    uncompressedSize?: number;
  };
}
```

### Constants

```typescript
// Protocol
export const PROTOCOL_VERSION = 1;
export const MAGIC_MESSAGE = 0x534f424d; // 'SOBM'
export const MAGIC_BATCH = 0x534f4231; // 'SOB1'

// Limits
export const MAX_CONTENT_CHARS = 280;
export const MAX_CONTENT_BYTES = 1120;
export const MAX_NONCE = 65535;
export const MAX_AUTHORS_PER_BATCH = 256;
export const BLOB_SIZE_LIMIT = 131072; // 128KB

// Signatures
export const BLS_PUBKEY_SIZE = 48;
export const BLS_SIGNATURE_SIZE = 96;
export const ECDSA_SIGNATURE_SIZE = 65;
```

### Error Handling

```typescript
import {
  SocialBlobsError,
  ContentTooLongError,
  InvalidMagicError,
  BatchOverflowError,
} from '@socialblobs/core';

try {
  const encoded = encodeMessage(message);
} catch (error) {
  if (error instanceof ContentTooLongError) {
    console.error('Message content exceeds limits');
  } else if (error instanceof SocialBlobsError) {
    console.error(`Protocol error: ${error.code}`);
  }
}
```

## Test Vectors

The package includes comprehensive test vectors for implementation validation:

- **`tests/vectors/messages.json`** - 10 message encoding test cases
- **`tests/vectors/batches.json`** - 9 batch encoding test cases
- **`tests/vectors/errors.json`** - 20 error handling test cases

Use these vectors to validate your implementation:

```typescript
import { readFileSync } from 'fs';
import { decodeMessage } from '@socialblobs/core';

const vectors = JSON.parse(readFileSync('tests/vectors/messages.json', 'utf-8'));

for (const vector of vectors.vectors) {
  const message = {
    author: vector.input.author,
    timestamp: vector.input.timestamp,
    nonce: vector.input.nonce,
    content: vector.input.content,
    signatureType: vector.input.signatureType,
  };

  const encoded = encodeMessage(message);
  const decoded = decodeMessage(encoded);

  console.assert(decoded.content === message.content);
}
```

## Protocol Specification

This implementation follows the Social-Blobs protocol specification:

- **Message Format**: 33-byte header + variable content + signature
- **Batch Format**: Header + author table + compressed messages
- **Compression**: Zstd with trained 32KB dictionary
- **Signatures**: BLS12-381 (aggregatable) or ECDSA (wallet-compatible)

For full protocol details, see `specs/002-protocol-spec/`.

## Compression

The library includes a trained Zstd dictionary optimized for social media content, achieving 5-10x
compression ratios on batches of 100+ messages.

```typescript
import { compress, decompress, compressionRatio, loadDictionary } from '@socialblobs/core';

// Load bundled dictionary
const dict = await loadDictionary();

// Compress data
const compressed = await compress(data, { dictionary: dict });
console.log(`Compression ratio: ${compressionRatio(data, compressed)}x`);

// Decompress
const decompressed = await decompress(compressed, { dictionary: dict });
```

## Performance

Benchmarks on Apple M1 Pro:

| Operation           | Time   | Throughput      |
| ------------------- | ------ | --------------- |
| Encode message      | 15 μs  | 66,000/sec      |
| Decode message      | 12 μs  | 83,000/sec      |
| Encode batch (100)  | 1.2 ms | 833 batches/s   |
| Decode batch (100)  | 0.9 ms | 1,111 batches/s |
| BLS sign            | 0.8 ms | 1,250/sec       |
| BLS verify          | 2.3 ms | 435/sec         |
| BLS aggregate (100) | 15 ms  | 6,667/sec       |

## Development

```bash
# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Run tests with coverage
npm run test:coverage

# Lint
npm run lint

# Clean build artifacts
npm run clean
```

## Documentation

- **[Quick Start Guide](specs/005-reference-impl/quickstart.md)** - Get started quickly
- **[Protocol Specification](specs/002-protocol-spec/)** - Full protocol details
- **[Compression Research](specs/003-compression-research/)** - Compression benchmarks
- **[Economic Analysis](specs/004-economic-analysis/)** - Cost modeling

## Contributing

Contributions are welcome! Please read the [constitution](.specify/memory/constitution.md) before
contributing.

## License

MIT

## Resources

- [Protocol Specification](specs/002-protocol-spec/)
- [EIP-4844 Blob Transactions](https://eips.ethereum.org/EIPS/eip-4844)
- [BLS12-381 Signatures](https://github.com/ethereum/bls12-381-tests)
- [Zstd Compression](https://facebook.github.io/zstd/)

## Acknowledgments

Built by [Wonderland](https://defi.sucks) as part of the Social-Blobs protocol research initiative.