npm package: @stabledata/silo
Description: JavaScript/TypeScript client for Silo asset management API with Express and Hono proxy helpers
Author: Stable Data
Latest version: 0.2.5

# @stabledata/silo

JavaScript/TypeScript client for the Silo asset management API. Handle file uploads, user management, and space organization with ease.

## Features

- 🗄️ **Asset Management** - Upload and organize files with automatic grouping
- 📸 **Live Photos** - Full support for iOS Live Photos (image + video pairs)
- 👥 **User Management** - Create and manage users
- 🏢 **Space Management** - Create and manage shared spaces
- 📊 **Upload Progress** - Real-time upload progress callbacks
- 🛡️ **Type Safe** - Full TypeScript support with comprehensive error types
- 🔄 **Batch Uploads** - Upload multiple files efficiently
- 🔌 **Proxy Helpers** - Express and Hono middleware for easy backend integration
- 🔧 **CLI Tool** - Command-line interface for admin operations

## Installation

```bash
npm install @stabledata/silo
```

## CLI Tool

The package includes a CLI tool for managing users, API keys, and spaces across environments.

### Installation

```bash
# Install globally
npm install -g @stabledata/silo

# Or run directly with npx
npx @stabledata/silo --help
```

### Usage

```bash
# Create an API key for your application
silo key create-app --name "Production API" --env prod

# Create a user with initial space
silo user create --email user@example.com --space my-space --env local

# List all users
silo user list --env staging --limit 100

# List keys for a user
silo key list --user user@example.com --env local

# Get user details
silo user get --email user@example.com --env prod

# Revoke a key
silo key revoke --key-id abc-123-def --env prod
```

### Environments

- **local** (default): Uses `http://localhost:8080` and default admin key
- **staging**: Uses staging API URL and fetches admin key from GCP Secret Manager
- **production**: Uses production API URL and fetches admin key from GCP Secret Manager

**Environment Variables:**
- `SILO_API_URL` or `SILO_HOST` - Override the API URL
- `SILO_ADMIN_KEY` or `ADMIN_API_KEY` - Your admin API key (preferred)
- `CREATE_ADMIN_USER_KEY` - Legacy admin key (still supported)

For staging/production, you need:
- Google Cloud SDK installed (`gcloud`)
- Authenticated: `gcloud auth login`
- Permission to access Secret Manager secrets

### CLI Commands

**Key Management:**
- `silo key create-app` - Create API key for your application
- `silo key list --user <email>` - List key metadata (ID, name, usage stats)
- `silo key rotate --name <name>` - Create new admin key and show instructions to revoke old one
- `silo key revoke --key-id <id>` - Revoke a key

**User Management:**
- `silo user create --email <email> --space <id>` - Create user with space
- `silo user list` - List all users
- `silo user get --email <email>` - Get user details

> **Note:** `create-admin` is deprecated, use `create-app` instead

### Key Security

**Important:** API keys are only shown once when created, then permanently hashed. You cannot retrieve the key value later.

- ✅ **When creating**: The full key is displayed - save it immediately
- ✅ **When listing**: Only metadata is shown (ID, name, created date, usage count)
- ❌ **Cannot retrieve**: Once created, the actual key value is never shown again

If you lose a key, you must create a new one and revoke the old one. Use `silo key rotate` to safely rotate keys.

## Quick Start

### For Backend/Proxy Apps (Recommended)

```typescript
import express from 'express';
import { createSiloMiddleware } from '@stabledata/silo/proxy/express';

const app = express();

// After your auth middleware
app.use('/api/silo', createSiloMiddleware({
  baseUrl: process.env.SILO_API_URL!,
  adminToken: process.env.SILO_ADMIN_TOKEN
}));

// Upload endpoint
app.post('/api/silo/upload', async (req, res) => {
  const results = await req.silo.uploadFiles('my-space', files);
  res.json(results);
});
```

See [Express Example](./examples/express-proxy/) or [Hono Example](./examples/hono-proxy/) for complete implementations.

### Direct Client Usage

```typescript
import { SiloClient } from '@stabledata/silo';

const client = new SiloClient({
  baseUrl: 'https://your-silo-api.com',
  userId: 'your-user-id',
  apiKey: 'user-api-key'
});

// Upload files
const results = await client.uploadFiles('my-space', files);
```

## Authentication Strategy

Silo uses a two-tier authentication model:

### 1. Admin Bearer Tokens
- Used for **user provisioning** (creating users)
- Should only exist on your backend server
- **Never expose to frontend clients**

```typescript
import { createAdminClient } from '@stabledata/silo/proxy/express';

const adminClient = createAdminClient({
  baseUrl: process.env.SILO_API_URL!,
  adminToken: process.env.SILO_ADMIN_TOKEN!
});

// Create user during signup
const siloUser = await adminClient.createUser(
  'user@example.com',
  'space-123',
  'User Space'
);

// Store siloUser.api_key in your database for this user
```

### 2. User API Keys
- Each user gets their own API key (returned from `createUser`)
- Scoped to that user's spaces and permissions
- Can be safely used in authenticated requests

```typescript
const userClient = new SiloClient({
  baseUrl: process.env.SILO_API_URL!,
  userId: user.id,
  apiKey: user.siloApiKey  // From your database
});
```

### Permission Model

API keys support fine-grained permissions:

- **Full Access**: All operations on all spaces
- **Space-Scoped**: Restricted to specific spaces
- **Operation-Scoped**: Upload-only, read-only, etc.

```typescript
// Create API key with custom permissions
const apiKey = await client.createApiKey({
  name: 'Upload-only key',
  permissions: {
    spaces: {
      allowed_spaces: ['vacation-2024'],
      upload: true,
      download: false,
      delete: false,
      list_assets: true,
      manage_spaces: false
    }
  }
});
```

## Integration Patterns

### Backend Proxy (Recommended)

Your backend handles authentication and proxies Silo requests:

```
Frontend → Your Backend (Auth) → Silo API
```

**Benefits:**
- Admin credentials never exposed
- Centralized auth logic
- Easy to add custom business logic
- Works with any auth system (JWT, sessions, etc.)

**Examples:**
- [Express Proxy](./examples/express-proxy/README.md) - Traditional Node.js apps
- [Hono Proxy](./examples/hono-proxy/README.md) - Modern/edge deployments

### Direct Client (Mobile/Desktop Apps)

Apps use user API keys directly:

```
Mobile App → Silo API (with user API key)
```

**Setup:**
1. User signs up via your backend
2. Backend creates Silo user with admin token
3. Backend returns user API key to app
4. App stores key securely (Keychain/KeyStore)
5. App uses key for all Silo operations

```typescript
// In your mobile app
const client = new SiloClient({
  baseUrl: 'https://silo.your-api.com',
  userId: currentUser.id,
  apiKey: secureStorage.get('siloApiKey')
});
```

## API Reference

### SiloClient

#### Constructor

```typescript
new SiloClient(config: SiloClientConfig)

interface SiloClientConfig {
  baseUrl: string;           // Your Silo API base URL
  userId: string;            // Current user ID
  apiKey?: string;           // User API key (for user operations)
  bearerToken?: string;      // Admin token (for admin operations)
}
```

### User Management

#### createUser(email, spaceId?, spaceName?)

Create a new user with initial space (requires admin token).

```typescript
const user = await adminClient.createUser(
  'user@example.com',
  'user-space-123',
  'My Photos'
);
// user.api_key - Store this in your database!
```

#### getUserByEmail(email)

Look up an existing user by email.

```typescript
const user = await client.getUserByEmail('user@example.com');
```

### Space Management

#### createSpace(spaceId, name?)

Create a new space for organizing files.

```typescript
const space = await client.createSpace('vacation-2024', 'Vacation Photos 2024');
```

**Space ID Requirements:**
- 3-63 characters
- Lowercase letters, numbers, and hyphens only
- Must start and end with alphanumeric characters

#### listSpaces()

Get all spaces the current user has access to.

```typescript
const spaces = await client.listSpaces();
```

#### updateSpace(request)

Update space name and manage members.

```typescript
await client.updateSpace({
  space_id: 'vacation-2024',
  name: 'Updated Name',
  members: [
    {
      email: 'friend@example.com',
      role: 'member',
      operation: 'create'
    }
  ]
});
```

### File Upload

#### uploadFile(spaceId, file, onProgress?)

Upload a single file.

```typescript
const results = await client.uploadFile('my-space', file, (progress) => {
  console.log(`${progress.message} - ${progress.progress}%`);
});
```

#### uploadFiles(spaceId, files, onProgress?)

Upload multiple files with automatic asset grouping.

```typescript
const files = [
  new File([/* data */], 'photo.jpg'),
  new File([/* data */], 'photo.mov'),  // Same basename = grouped asset
  new File([/* data */], 'another.jpg')
];

const results = await client.uploadFiles('my-space', files, (progress) => {
  console.log(`Stage: ${progress.stage}`);
  console.log(`File: ${progress.fileName}`);
  console.log(`Progress: ${progress.progress}%`);
});
```

**Asset Grouping Logic:**
- Files with the same basename (different extensions) are grouped together
- JPG/HEIC + MOV with same basename automatically detected as Live Photos
- Single files become `SINGLE` assets
- Multiple ungrouped files become `BATCH` assets

#### uploadLivePhoto(spaceId, imageFile, videoFile, onProgress?)

Upload a Live Photo (image + video pair).

```typescript
const imageFile = new File([/* data */], 'IMG_001.heic');
const videoFile = new File([/* data */], 'IMG_001.mov');

const results = await client.uploadLivePhoto('my-space', imageFile, videoFile);
// results[0].hasLiveVideo === true
// results[0].assetType === 'LIVE_PHOTO'
```

### Asset Retrieval

#### listAssets(spaceId, thumbnailSize?)

List all assets in a space with thumbnails. **Always returns complete files array** per Silo specifications.

```typescript
const assetList = await client.listAssets('my-space', 'square');

assetList.assets.forEach(asset => {
  // Access complete files array (ALWAYS included)
  asset.files.forEach(file => {
    console.log(`${file.fileName} (${file.fileRole})`);
    console.log(`Direct URL: ${file.signedUrl}`);
  });

  // Use requestedFormatURL for gallery thumbnails
  if (asset.requestedFormatURL) {
    console.log(`Thumbnail: ${asset.requestedFormatURL}`);
  }
});
```

**Available Thumbnail Sizes:**
- `square` - 150x150px (grids, profile pics)
- `small` - 320x240px (mobile list views)
- `medium` - 800x600px (desktop previews)
- `large` - 1920x1440px (full-screen preview)

#### getAssetFiles(spaceId, assetId, thumbnailSize?)

Get files for a specific asset with optional thumbnail generation.

```typescript
const asset = await client.getAssetFiles('my-space', 'asset-uuid', 'square');
console.log(asset.files); // All files including thumbnail
```

### Asset Management

#### deleteAsset(spaceId, assetId)

Delete an asset and all its files.

```typescript
await client.deleteAsset('my-space', 'asset-uuid');
```

#### deleteFile(spaceId, fileId)

Delete a specific file from an asset.

```typescript
await client.deleteFile('my-space', 'file-uuid');
```

#### copyAsset(fromSpaceId, toSpaceId, assetId)

Copy an asset to a different space.

```typescript
const result = await client.copyAsset('space-1', 'space-2', 'asset-uuid');
```

#### moveAsset(fromSpaceId, toSpaceId, assetId)

Move an asset to a different space.

```typescript
const result = await client.moveAsset('space-1', 'space-2', 'asset-uuid');
```

### API Key Management

#### createApiKey(request)

Create a new API key with custom permissions.

```typescript
const apiKey = await client.createApiKey({
  name: 'Mobile App Key',
  permissions: {
    spaces: {
      upload: true,
      download: true,
      delete: false,
      list_assets: true,
      manage_spaces: false
    }
  },
  expiresAt: '2025-12-31T23:59:59Z'
});

// apiKey.apiKey contains the full key (only returned once!)
```

#### listApiKeys()

List all API keys for the current user.

```typescript
const keys = await client.listApiKeys();
```

#### updateApiKey(keyId, request)

Update API key properties.

```typescript
await client.updateApiKey('key-id', {
  name: 'Updated Name',
  isActive: false
});
```

#### deleteApiKey(keyId)

Delete/revoke an API key.

```typescript
await client.deleteApiKey('key-id');
```

## Error Handling

The client provides typed error classes for different failure scenarios:

```typescript
import {
  SpaceNotFoundError,
  QuotaExceededError,
  InvalidFileError
} from '@stabledata/silo';

try {
  await client.uploadFile('nonexistent-space', file);
} catch (error) {
  if (error instanceof SpaceNotFoundError) {
    console.log('Space does not exist');
  } else if (error instanceof QuotaExceededError) {
    console.log('Storage quota exceeded');
  } else if (error instanceof InvalidFileError) {
    console.log('Invalid file format');
  }
}
```

### Error Types

- `MissingHeaderError` - Required header missing
- `InvalidUserIdError` - Invalid user ID format
- `SpaceAccessDeniedError` - No access to space
- `SpaceNotFoundError` - Space does not exist
- `QuotaExceededError` - Storage quota exceeded
- `InvalidFileError` - Invalid file format/size
- `FileProcessingError` - Server-side processing error
- `StorageError` - Storage backend error
- `DatabaseError` - Database operation error
- `AssetNotFoundError` - Asset does not exist
- `FileNotFoundError` - File does not exist
- `AssetCopyError` - Asset copy/move failed

## Proxy Helpers

### Express

```typescript
import express from 'express';
import {
  createSiloMiddleware,
  createAdminClient,
  createUploadHandler,
  createListAssetsHandler
} from '@stabledata/silo/proxy/express';

const app = express();

// Admin client for user creation
const adminClient = createAdminClient({
  baseUrl: process.env.SILO_API_URL!,
  adminToken: process.env.SILO_ADMIN_TOKEN!
});

// User creation endpoint
app.post('/api/users', async (req, res) => {
  const siloUser = await adminClient.createUser(req.body.email);
  // Store siloUser.api_key in your database
  res.json({ user: siloUser });
});

// Protected routes with Silo middleware
app.use('/api/silo', authenticateUser);
app.use('/api/silo', createSiloMiddleware({
  baseUrl: process.env.SILO_API_URL!,
  getUserId: (req) => req.user.id,
  getApiKey: (req) => req.user.siloApiKey
}));

// Upload endpoint
app.post('/api/silo/upload',
  multer().array('files'),
  createUploadHandler({
    getSpaceId: (req) => req.body.spaceId
  })
);

// List assets endpoint
app.get('/api/silo/spaces/:spaceId/assets', createListAssetsHandler());
```

See [Express Example](./examples/express-proxy/) for full implementation.

### Hono

```typescript
import { Hono } from 'hono';
import {
  createSiloMiddleware,
  createAdminClient,
  createUploadHandler,
  createListAssetsHandler
} from '@stabledata/silo/proxy/hono';

const app = new Hono();

// Admin client for user creation
const adminClient = createAdminClient({
  baseUrl: process.env.SILO_API_URL!,
  adminToken: process.env.SILO_ADMIN_TOKEN!
});

// Protected routes
const api = new Hono();
api.use('*', authenticateUser);
api.use('*', createSiloMiddleware({
  baseUrl: process.env.SILO_API_URL!,
  getUserId: (c) => c.get('userId'),
  getApiKey: (c) => c.get('apiKey')
}));

api.post('/upload', createUploadHandler({
  getSpaceId: (c) => c.req.query('space')
}));

api.get('/spaces/:spaceId/assets', createListAssetsHandler());

app.route('/api/silo', api);
```

See [Hono Example](./examples/hono-proxy/) for full implementation including Cloudflare Workers deployment.

## TypeScript Types

The client exports comprehensive TypeScript types:

```typescript
import type {
  UserResponse,
  SpaceResponse,
  AssetUploadResult,
  UploadProgress,
  SiloClientConfig,
  AssetListItem,
  AssetFileInfo,
  ApiKeyResponse
} from '@stabledata/silo';
```

## Security Best Practices

### ✅ DO

1. **Keep admin tokens secure**
   - Store in environment variables
   - Never commit to version control
   - Only use on backend servers

2. **Store user API keys properly**
   - Encrypt in database
   - Use secure storage on mobile (Keychain/KeyStore)
   - Never log or expose in errors

3. **Use HTTPS in production**
   - Always use TLS for API communications
   - Validate SSL certificates

4. **Implement rate limiting**
   - Protect upload endpoints
   - Prevent abuse

5. **Validate inputs**
   - Check file sizes and types
   - Sanitize user-provided space IDs

### ❌ DON'T

1. **Never expose admin tokens**
   - Don't send to frontend
   - Don't include in client bundles
   - Don't log in production

2. **Don't trust client data**
   - Always verify user identity on backend
   - Don't use client-provided user IDs directly
   - Validate all inputs

3. **Don't return API keys unnecessarily**
   - Only return during user creation
   - Never include in list/get responses
   - Don't log API keys

## Examples

### Complete Backend Integration

See working examples:
- [Express Proxy Server](./examples/express-proxy/) - Traditional Node.js
- [Hono Proxy Server](./examples/hono-proxy/) - Modern/Edge/Cloudflare Workers

### Mobile App Pattern

```typescript
// 1. During signup (backend)
app.post('/signup', async (req, res) => {
  const siloUser = await adminClient.createUser(req.body.email);
  await db.users.create({
    ...req.body,
    siloApiKey: siloUser.api_key  // Store encrypted
  });
  res.json({ success: true });
});

// 2. In mobile app (after authentication)
const user = await yourAuth.getCurrentUser();
const siloClient = new SiloClient({
  baseUrl: 'https://silo.your-api.com',
  userId: user.id,
  apiKey: await secureStorage.get('siloApiKey')
});

// 3. Upload photos
const results = await siloClient.uploadFiles('my-space', photos);
```

### Web App Pattern

```typescript
// Frontend calls your backend proxy
const uploadToSilo = async (files: File[]) => {
  const formData = new FormData();
  files.forEach(f => formData.append('files', f));

  const response = await fetch('/api/silo/upload', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${userToken}` // Your auth
    },
    body: formData
  });

  return response.json();
};
```

## Test Application

This package includes a complete React test application:

```bash
cd test-app
npm install
npm run dev
```

The test app demonstrates:
- User creation and lookup
- Space management
- File upload with drag & drop
- Live Photo upload
- Real-time progress tracking
- Error handling

## License

MIT

## Support

For issues and feature requests, please visit the [GitHub repository](https://github.com/stabledata/silo).