npm package: @fatagnus/dink-sync
Description: Offline-first sync SDK for Dink edge platform with Effect.ts
Author: Ozan Turksever
Latest version: 2.32.0

# @fatagnus/dink-sync

Offline-first sync SDK for Dink edge platform with Effect.ts.

This SDK powers the **Offline-First Edge** type - edges that work offline and automatically sync when connectivity is restored.

| Edge Type | SDK | Use Case |
|-----------|-----|----------|
| **Lite Edge** | `@fatagnus/dink-sdk` | Always-connected IoT, RPC services |
| **Offline-First Edge** | ✅ This SDK | Offline data with auto-sync |
| **Full Edge** | Not yet available | Local NATS services (planned) |

## Installation

```bash
npm install @fatagnus/dink-sync
# or
pnpm add @fatagnus/dink-sync
```

## Features

- **Offline-first**: Queue changes locally and sync when online
- **CRDT-based**: Conflict-free replicated data types for automatic conflict resolution
- **Real-time sync**: Bidirectional sync with dinkd server via NATS
- **React hooks**: Ready-to-use hooks for React applications
- **Persistence**: Pluggable persistence providers (memory, PGlite)
- **Type-safe**: Full TypeScript support with strict typing
- **Typed Client Factory**: Generate type-safe collection access from Convex schema

## Quick Start — ⭐ Typed Client (STRONGLY PREFERRED)

**Always use typed clients** generated from your Convex schema. This is the only recommended approach for production applications.

> ⚠️ Do not use the low-level sync engine or generic collection APIs in production. They should only be used for quick prototyping.

### 1. Define Your Schema

```typescript
// convex/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";

export default defineSchema({
  tasks: defineTable({
    text: v.string(),
    isCompleted: v.boolean(),
    priority: v.optional(v.number()),
  }),
  users: defineTable({
    name: v.string(),
    email: v.string(),
  }),
});
```

### 2. Generate Typed Client

```bash
# Generate TypeScript types, validators, and typed client
dink codegen --convex-schema ./convex --convex-output ./src/generated --zod
```

### 3. Use the Typed Client

```typescript
import { offlineEdge } from '@fatagnus/dink-sync';
import { PGlitePersistence } from '@fatagnus/dink-sync/persistence';
import { createTypedClient } from './generated';

// Initialize the offline edge client
const edge = offlineEdge.create({
  persistence: new PGlitePersistence(),
  config: {
    serverUrl: 'nats://localhost:4222',
    apiKey: 'your-edge-api-key',
  },
});
await edge.init();

// Create the typed client - this is the recommended way!
const db = createTypedClient(edge.get());

// Type-safe collection access with full IDE autocompletion
const task = await db.tasks.insert({
  text: 'Buy milk',
  isCompleted: false,
});
console.log(`Created task: ${task.id}`);

// List all tasks
const allTasks = await db.tasks.list();

// Update a task
const updated = await db.tasks.update(task.id, { isCompleted: true });

// Delete a task
await db.tasks.delete(task.id);

// Validation is automatic!
try {
  await db.tasks.insert({ text: '', isCompleted: false });
} catch (err) {
  console.error('Validation failed:', err.message); // "text is required"
}
```

### Why Typed Client?

| Feature | Typed Client | Generic Collection |
|---------|--------------|--------------------|
| Type Safety | ✅ Compile-time checks | ⚠️ Runtime only |
| IDE Support | ✅ Full autocompletion | ⚠️ Limited |
| Validation | ✅ Automatic | ❌ Manual setup |
| Refactoring | ✅ Safe renames | ⚠️ String-based |

## Alternative: Low-Level Sync Engine ⚠️ Advanced/Prototyping Only

For advanced use cases or quick prototyping, you can use the sync engine directly. **Migrate to typed clients before production:**

```typescript
import { createSyncEngine, NatsTransport } from '@fatagnus/dink-sync/client';

// Create transport and engine
const transport = new NatsTransport({
  serverUrl: 'nats://localhost:4222',
  apiKey: 'your-edge-api-key',
  appId: 'your-app-id',
  edgeId: 'your-edge-id',
});

const engine = createSyncEngine({
  serverUrl: 'nats://localhost:4222',
  apiKey: 'your-edge-api-key',
  transport,
});

// Connect and register documents
await engine.connect();
const actor = await engine.registerDocument('tasks', 'task-1');

// Queue changes
actor.queueChange(new Uint8Array([1, 2, 3]));

// Listen for sync events
engine.onSyncComplete((event) => {
  console.log(`Synced ${event.documentId}`);
});
```

## React Integration

```tsx
import { OfflineEdgeProvider, useCollection } from '@fatagnus/dink-sync/react';

function App() {
  return (
    <OfflineEdgeProvider config={{ serverUrl, apiKey, edgeId }}>
      <TaskList />
    </OfflineEdgeProvider>
  );
}

function TaskList() {
  const { items, insert, update, delete: remove } = useCollection('tasks');
  // ...
}
```

## Testing

### Unit Tests

Run unit tests (uses mock transport):

```bash
pnpm test
```

### E2E Tests

E2E tests use testcontainers to spin up a real dinkd server.

### Browser Tests

Browser tests verify PGlite persistence with IndexedDB in a real browser environment. 

#### Prerequisites

1. **Docker**: Install and ensure Docker daemon is running

2. **dinkd image**: Build the dinkd Docker image from the project root:
   ```bash
   # From the project root directory
   docker build -t dinkd:latest .
   ```

#### Running E2E Tests

```bash
pnpm test:e2e
```

#### Running Browser Tests

**Prerequisites:**

1. **Playwright browsers**: Install Chromium for browser testing:
   ```bash
   npx playwright install chromium
   ```

**Run browser tests:**

```bash
pnpm test:browser
```

**Browser test coverage:**

- Data persistence with IndexedDB backend
- Data persistence across page reloads/sessions
- Large dataset handling (1000+ documents)
- Concurrent read/write operations
- Binary data integrity
- Special character handling in document IDs

#### E2E Test Coverage

The E2E tests verify:

- Edge connection to real dinkd server
- Document registration and sync operations
- External update subscription
- Concurrent sync from multiple edges
- Reconnection handling
- Error handling and event emission

#### Test Environment Variables

| Variable | Description | Default |
|----------|-------------|--------|
| `DINK_TEST_IMAGE` | Docker image for dinkd | `dinkd:latest` |

#### Troubleshooting Browser Tests

**Playwright not installed:**
- Run `npx playwright install chromium` to install the browser

**Tests timing out:**
- Large dataset tests may take longer; timeout is set to 60 seconds
- Consider running browser tests separately from unit tests

**IndexedDB errors:**
- Ensure you're running in a supported browser (Chromium)
- Check for quota limits in browser settings

#### Troubleshooting E2E Tests

**Container startup timeout**:
- Ensure Docker is running
- Check that the `dinkd:latest` image exists: `docker images | grep dinkd`
- Increase startup timeout in test configuration if needed

**Connection refused**:
- Container may not be fully started
- Check container logs for errors

**Port conflicts**:
- Testcontainers automatically maps to random ports, but conflicts can occur
- Stop any running dinkd containers manually if needed

## API Reference

### SyncEngine

- `connect()`: Connect to dinkd server
- `disconnect()`: Disconnect from server
- `destroy()`: Clean up all resources
- `registerDocument(collection, docId)`: Register a document for sync
- `unregisterDocument(collection, docId)`: Unregister a document
- `getConnectionState()`: Get current connection state
- `onStateChange(callback)`: Subscribe to connection state changes
- `onSyncStarted(callback)`: Subscribe to sync start events
- `onSyncComplete(callback)`: Subscribe to sync completion events
- `onSyncError(callback)`: Subscribe to sync error events
- `onSyncRejected(callback)`: Subscribe to sync rejection events
- `goOffline()`: Manually force offline mode
- `goOnline()`: Exit manual offline mode
- `discardLocalChanges(collection, docId)`: Discard pending changes for a document
- `forcePush(collection, docId)`: Force push local state to server

### DocumentActor

- `queueChange(delta)`: Queue a change for sync
- `hasPendingChanges()`: Check if there are pending changes
- `onPendingChange(callback)`: Subscribe to pending state changes
- `getStateVector()`: Get current state vector
- `setStateVector(vector)`: Set state vector
- `applyExternalUpdate(update)`: Apply an external update
- `onExternalUpdate(callback)`: Subscribe to external updates

### ConnectionState

- `Offline`: Not connected to server
- `Connecting`: Connection in progress
- `Online`: Connected and ready
- `Reconnecting`: Connection lost, attempting to reconnect

## License

Apache-2.0