npm package: @lionhummer/safe-api
Description: Flatfile Safe API client with streaming capabilities
Author: Flatfile
Latest version: 0.1.4

# @flatfile/safe-api

A safer, more efficient version of @flatfile/api that adds streaming capabilities while maintaining all original functionality.

## Installation

```bash
npm install @flatfile/safe-api
```

## Usage

This package provides a safer version of @flatfile/api with additional streaming capabilities while maintaining all existing functionality and the familiar API interface.

### Basic Usage

```typescript
import api from '@flatfile/safe-api';

// Use exactly like the original @flatfile/api
const records = await api.records.get(workbookId);
```

### Streaming Capabilities

The streaming API provides four different methods for working with records:

#### 1. Get Records (Full Data with Collection Support)

```typescript
import api from '@flatfile/safe-api';
import { Collection, Item } from '@flatfile/safe-api';

// Get all records at once (default behavior)
const records = await api.records.stream.get({ 
  workbookId: 'your-workbook-id',
  includeMetadata: true,
  includeConfig: true 
});

// Or stream records in chunks for large datasets
for await (const collection of api.records.stream.get({ 
  workbookId: 'your-workbook-id',
  includeMetadata: true,
  includeConfig: true,
  stream: true
})) {
  // collection is a Collection<Item> with change tracking
  collection.each((item: Item<any>) => {
    console.log('Full record:', item.toJSON());
  });
}
```

#### 2. Simple Records (Clean Data)

```typescript
import api from '@flatfile/safe-api';

// Get all records at once in simplified format
const records = await api.records.stream.simple({ 
  workbookId: 'your-workbook-id' 
});

// Or stream records one by one for large datasets
for await (const record of api.records.stream.simple({ 
  workbookId: 'your-workbook-id',
  stream: true
})) {
  console.log('Clean record:', record);
  // record has no internal __k, __m prefixes
}
```

#### 3. Update Records (With Change Tracking)

```typescript
import api from '@flatfile/safe-api';
import { Collection, Item } from '@flatfile/safe-api';

// Create a collection of records
const collection = new Collection([
  new Item({ id: '1', name: 'John' }),
  new Item({ id: '2', name: 'Jane' })
]);

// Make changes
collection.each((item: Item<any>) => {
  item.toJSON().name = item.toJSON().name.toUpperCase();
});

// Update records with change tracking
await api.records.stream.update(
  { 
    workbookId: 'your-workbook-id',
    truncate: false,
    snapshot: true
  },
  collection
);
```

#### 4. Raw Upsert (Direct JSONL)

```typescript
import api from '@flatfile/safe-api';

// Raw upsert of records
const records = [
  { id: '1', name: 'John' },
  { id: '2', name: 'Jane' }
];

await api.records.stream.raw(
  { 
    workbookId: 'your-workbook-id',
    truncate: true
  },
  records
);
```

## API Reference

### api.records.stream.get(options?: StreamRecordsQuery)

Gets or streams records as Collection<Item> with full metadata and change tracking support.

Options:
- `workbookId`: The ID of the workbook
- `sheetId`: The ID of the sheet (alternative to workbookId)
- `stream`: When true, returns AsyncGenerator for streaming records. When false (default), returns all records at once
- `includeMetadata`: Include record metadata
- `includeConfig`: Include configuration
- `includeSheet`: Include sheet information
- `includeSheetSlug`: Include sheet slug
- `noContext`: Exclude context information

Returns: 
- When `stream: false` (default): Promise<Collection<Item<any>>>
- When `stream: true`: AsyncGenerator<Collection<Item<any>>>

### api.records.stream.simple(options?: StreamRecordsQuery)

Gets or streams records in a simplified format without internal metadata.

Options:
- `workbookId`: The ID of the workbook
- `sheetId`: The ID of the sheet (alternative to workbookId)
- `stream`: When true, returns AsyncGenerator for streaming records. When false (default), returns all records at once

Returns:
- When `stream: false` (default): Promise<SimpleRecord[]>
- When `stream: true`: AsyncGenerator<SimpleRecord>

### api.records.stream.update(options: StreamRecordsPatchQuery, records: Collection<Item<any>>)

Updates records using JSONL format with change tracking support.

Options:
- `workbookId`: The ID of the workbook
- `sheetId`: The ID of the sheet (alternative to workbookId)
- `truncate`: Whether to truncate existing records (default: false)
- `snapshot`: Create a snapshot before update (default: false)
- `silent`: Suppress notifications (default: false)

Returns: Promise<{ success: true }>

### api.records.stream.raw(options: StreamRecordsPatchQuery, records: Array<Record<string, Primitive>>)

Raw upsert of records using JSONL format with GZIP compression.

Options:
- `workbookId`: The ID of the workbook
- `sheetId`: The ID of the sheet (alternative to workbookId)
- `truncate`: Whether to truncate existing records (default: false)
- `snapshot`: Create a snapshot before update (default: false)
- `silent`: Suppress notifications (default: false)

Returns: Promise<{ success: true }>

## Types

### Item<T>

A wrapper class for records that provides change tracking and record management.

```typescript
class Item<T> {
  constructor(data: T);
  toJSON(): T;                 // Get current state
  isDeleted(): boolean;        // Check if marked for deletion
  delete(): void;              // Mark for deletion
  changeset(): T;              // Get changes
  commit(): void;              // Commit changes
}
```

### SimpleRecord

```typescript
interface SimpleRecord {
  id?: string;
  metadata?: Record<string, any>;
  [key: string]: string | number | boolean | null | undefined | Record<string, any>;
}
```

### StreamRecordsQuery

```typescript
interface StreamRecordsQuery {
  sheetId?: string;
  workbookId?: string;
  stream?: boolean;           // When true, returns AsyncGenerator for streaming
  includeMetadata?: boolean;
  includeConfig?: boolean;
  includeSheet?: boolean;
  includeSheetSlug?: boolean;
  noContext?: boolean;
}
```

## Change Tracking

The `Item` class and `Collection` support provide robust change tracking:

```typescript
import { Collection, Item } from '@flatfile/safe-api';

// Create a collection
const collection = new Collection([
  new Item({ id: '1', name: 'John' })
]);

// Make changes
collection.each((item: Item<any>) => {
  const record = item.toJSON();
  record.name = 'Johnny';
  // Changes are tracked automatically
});

// Get only changed records
const changes = collection.changes();

// Commit changes after successful update
changes.each((item: Item<any>) => item.commit());
```

## Error Handling

All streaming methods will throw errors in the following cases:

1. Network errors (failed requests)
2. Invalid workbook or sheet IDs
3. Permission issues
4. Invalid record format in updates
5. Empty updates when not using truncate mode
6. Temporary record deletion issues

Example error handling:

```typescript
try {
  for await (const record of api.records.stream.simple({ workbookId })) {
    await processRecord(record);
  }
} catch (error) {
  console.error('Streaming error:', error);
}
```

## License

MIT