Skill: okaris/javascript-sdk (version 0.1.5)

--- name: javascript-sdk description: "JavaScript/TypeScript SDK for inference.sh - run AI apps, build agents, integrate 150+ models. Package: @inferencesh/sdk (npm install). Full TypeScript support, streaming, file uploads. Build agents with template or ad-hoc patterns, tool builder API, skills, human approval. Use for: JavaScript integration, TypeScript, Node.js, React, Next.js, frontend apps. Triggers: javascript sdk, typescript sdk, npm install, node.js api, js client, react ai, next.js ai, frontend sdk, @inferencesh/sdk, typescript agent, browser sdk, js integration" allowed-tools: Bash(npm *), Bash(npx *), Bash(node *), Bash(pnpm *), Bash(yarn *) --- # JavaScript SDK Build AI applications with the [inference.sh](https://inference.sh) JavaScript/TypeScript SDK. ![JavaScript SDK](https://cloud.inference.sh/app/files/u/4mg21r6ta37mpaz6ktzwtt8krr/01kgvftjwhby36trvaj66bwzcf.jpeg) ## Quick Start ```bash npm install @inferencesh/sdk ``` ```typescript import { inference } from '@inferencesh/sdk'; const client = inference({ [REDACTED_SECRET]' }); // Run an AI app const result = await client.run({ app: 'infsh/flux-schnell', input: { prompt: 'A sunset over mountains' } }); console.log(result.output); ``` ## Installation ```bash npm install @inferencesh/sdk # or yarn add @inferencesh/sdk # or pnpm add @inferencesh/sdk ``` **Requirements:** Node.js 18.0.0+ (or modern browser with fetch) ## Authentication ```typescript import { inference } from '@inferencesh/sdk'; // Direct API key const client = inference({ [REDACTED_SECRET]' }); // From environment variable (recommended) const client = inference({ [REDACTED_SECRET] }); // For frontend apps (use proxy) const client = inference({ proxyUrl: '/api/inference/proxy' }); ``` Get your API key: Settings → API Keys → Create API Key ## Running Apps ### Basic Execution ```typescript const result = await client.run({ app: 'infsh/flux-schnell', input: { prompt: 'A cat astronaut' } }); console.log(result.status); // "completed" console.log(result.output); // Output data ``` ### Fire and Forget ```typescript const task = await client.run({ app: 'google/veo-3-1-fast', input: { prompt: 'Drone flying over mountains' } }, { wait: false }); console.log(`Task ID: ${task.id}`); // Check later with client.getTask(task.id) ``` ### Streaming Progress ```typescript const stream = await client.run({ app: 'google/veo-3-1-fast', input: { prompt: 'Ocean waves at sunset' } }, { stream: true }); for await (const update of stream) { console.log(`Status: ${update.status}`); if (update.logs?.length) { console.log(update.logs.at(-1)); } } ``` ### Run Parameters | Parameter | Type | Description | |-----------|------|-------------| | `app` | string | App ID (namespace/name@version) | | `input` | object | Input matching app schema | | `setup` | object | Hidden setup configuration | | `infra` | string | 'cloud' or 'private' | | `session` | string | Session ID for stateful execution | | `session_timeout` | number | Idle timeout (1-3600 seconds) | ## File Handling ### Automatic Upload ```typescript const result = await client.run({ app: 'image-processor', input: { image: '/path/to/image.png' // Auto-uploaded } }); ``` ### Manual Upload ```typescript // Basic upload const file = await client.uploadFile('/path/to/image.png'); // With options const file = await client.uploadFile('/path/to/image.png', { filename: 'custom_name.png', contentType: 'image/png', public: true }); const result = await client.run({ app: 'image-processor', input: { image: file.uri } }); ``` ### Browser File Upload ```typescript const input = document.querySelector('input[type="file"]'); const file = await client.uploadFile(input.files[0]); ``` ## Sessions (Stateful Execution) Keep workers warm across multiple calls: ```typescript // Start new session const result = await client.run({ app: 'my-app', input: { action: 'init' }, session: 'new', session_timeout: 300 // 5 minutes }); const sessionId = result.session_id; // Continue in same session const result2 = await client.run({ app: 'my-app', input: { action: 'process' }, session: sessionId }); ``` ## Agent SDK ### Template Agents Use pre-built agents from your workspace: ```typescript const agent = client.agent('my-team/support-agent@latest'); // Send message const response = await agent.sendMessage('Hello!'); console.log(response.text); // Multi-turn conversation const response2 = await agent.sendMessage('Tell me more'); // Reset conversation agent.reset(); // Get chat history const chat = await agent.getChat(); ``` ### Ad-hoc Agents Create custom agents programmatically: ```typescript import { tool, string, number, appTool } from '@inferencesh/sdk'; // Define tools const calculator = tool('calculate') .describe('Perform a calculation') .param('expression', string('Math expression')) .build(); const imageGen = appTool('generate_image', 'infsh/flux-schnell@latest') .describe('Generate an image') .param('prompt', string('Image description')) .build(); // Create agent const agent = client.agent({ core_app: { ref: 'infsh/claude-sonnet-4@latest' }, system_prompt: 'You are a helpful assistant.', tools: [calculator, imageGen], temperature: 0.7, max_tokens: 4096 }); const response = await agent.sendMessage('What is 25 * 4?'); ``` ### Available Core Apps | Model | App Reference | |-------|---------------| | Claude Sonnet 4 | `infsh/claude-sonnet-4@latest` | | Claude 3.5 Haiku | `infsh/claude-haiku-35@latest` | | GPT-4o | `infsh/gpt-4o@latest` | | GPT-4o Mini | `infsh/gpt-4o-mini@latest` | ## Tool Builder API ### Parameter Types ```typescript import { string, number, integer, boolean, enumOf, array, obj, optional } from '@inferencesh/sdk'; const name = string('User\'s name'); const age = integer('Age in years'); const score = number('Score 0-1'); const active = boolean('Is active'); const priority = enumOf(['low', 'medium', 'high'], 'Priority'); const tags = array(string('Tag'), 'List of tags'); const address = obj({ street: string('Street'), city: string('City'), zip: optional(string('ZIP')) }, 'Address'); ``` ### Client Tools (Run in Your Code) ```typescript const greet = tool('greet') .display('Greet User') .describe('Greets a user by name') .param('name', string('Name to greet')) .requireApproval() .build(); ``` ### App Tools (Call AI Apps) ```typescript const generate = appTool('generate_image', 'infsh/flux-schnell@latest') .describe('Generate an image from text') .param('prompt', string('Image description')) .setup({ model: 'schnell' }) .input({ steps: 20 }) .requireApproval() .build(); ``` ### Agent Tools (Delegate to Sub-agents) ```typescript import { agentTool } from '@inferencesh/sdk'; const researcher = agentTool('research', 'my-org/researcher@v1') .describe('Research a topic') .param('topic', string('Topic to research')) .build(); ``` ### Webhook Tools (Call External APIs) ```typescript import { webhookTool } from '@inferencesh/sdk'; const notify = webhookTool('slack', 'https://hooks.slack.com/...') .describe('Send Slack notification') .secret('SLACK_SECRET') .param('channel', string('Channel')) .param('message', string('Message')) .build(); ``` ### Internal Tools (Built-in Capabilities) ```typescript import { internalTools } from '@inferencesh/sdk'; const config = internalTools() .plan() .memory() .webSearch(true) .codeExecution(true) .imageGeneration({ enabled: true, appRef: 'infsh/flux@latest' }) .build(); const agent = client.agent({ core_app: { ref: 'infsh/claude-sonnet-4@latest' }, internal_tools: config }); ``` ## Streaming Agent Responses ```typescript const response = await agent.sendMessage('Explain quantum computing', { onMessage: (msg) => { if (msg.content) { process.stdout.write(msg.content); } }, onToolCall: async (call) => { console.log(`\n[Tool: ${call.name}]`); const result = await executeTool(call.name, call.args); agent.submitToolResult(call.id, result); } }); ``` ## File Attachments ```typescript // From file path (Node.js) import { readFileSync } from 'fs'; const response = await agent.sendMessage('What\'s in this image?', { files: [readFileSync('image.png')] }); // From base64 const response = await agent.sendMessage('Analyze this', { files: ['data:image/png;base64,iVBORw0KGgo...'] }); // From browser File object const input = document.querySelector('input[type="file"]'); const response = await agent.sendMessage('Describe this', { files: [input.files[0]] }); ``` ## Skills (Reusable Context) ```typescript const agent = client.agent({ core_app: { ref: 'infsh/claude-sonnet-4@latest' }, skills: [ { name: 'code-review', description: 'Code review guidelines', content: '# Code Review\n\n1. Check security\n2. Check performance...' }, { name: 'api-docs', description: 'API documentation', url: 'https://example.com/skills/api-docs.md' } ] }); ``` ## Server Proxy (Frontend Apps) For browser apps, proxy through your backend to keep API keys secure: ### Client Setup ```typescript const client = inference({ proxyUrl: '/api/inference/proxy' // No apiKey needed on frontend }); ``` ### Next.js Proxy (App Router) ```typescript // app/api/inference/proxy/route.ts import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs'; const route = createRouteHandler({ [REDACTED_SECRET] }); export const POST = route.POST; ``` ### Express Proxy ```typescript import express from 'express'; import { createProxyMiddleware } from '@inferencesh/sdk/proxy/express'; const app = express(); app.use('/api/inference/proxy', createProxyMiddleware({ [REDACTED_SECRET] })); ``` ### Supported Frameworks - Next.js (App Router & Pages Router) - Express - Hono - Remix - SvelteKit ## TypeScript Support Full type definitions included: ```typescript import type { TaskDTO, ChatDTO, ChatMessageDTO, AgentTool, TaskStatusCompleted, TaskStatusFailed } from '@inferencesh/sdk'; if (result.status === TaskStatusCompleted) { console.log('Done!'); } else if (result.status === TaskStatusFailed) { console.log('Failed:', result.error); } ``` ## Error Handling ```typescript import { RequirementsNotMetException, InferenceError } from '@inferencesh/sdk'; try { const result = await client.run({ app: 'my-app', input: {...} }); } catch (e) { if (e instanceof RequirementsNotMetException) { console.log('Missing requirements:'); for (const err of e.errors) { console.log(` - ${err.type}: ${err.key}`); } } else if (e instanceof InferenceError) { console.log('API error:', e.message); } } ``` ## Human Approval Workflows ```typescript const response = await agent.sendMessage('Delete all temp files', { onToolCall: async (call) => { if (call.requiresApproval) { const approved = await promptUser(`Allow ${call.name}?`); if (approved) { const result = await executeTool(call.name, call.args); agent.submitToolResult(call.id, result); } else { agent.submitToolResult(call.id, { error: 'Denied by user' }); } } } }); ``` ## CommonJS Support ```javascript const { inference, tool, string } = require('@inferencesh/sdk'); const client = inference({ [REDACTED_SECRET]' }); const result = await client.run({...}); ``` ## Reference Files - [Agent Patterns](references/agent-patterns.md) - Multi-agent, RAG, batch processing patterns - [Tool Builder](references/tool-builder.md) - Complete tool builder API reference - [Server Proxy](references/server-proxy.md) - Next.js, Express, Hono, Remix, SvelteKit setup - [Streaming](references/streaming.md) - Real-time progress updates and SSE handling - [File Handling](references/files.md) - Upload, download, and manage files - [Sessions](references/sessions.md) - Stateful execution with warm workers - [TypeScript](references/typescript.md) - Type definitions and type-safe patterns - [React Integration](references/react-integration.md) - Hooks, components, and patterns ## Related Skills ```bash # Python SDK npx skills add inference-sh/skills@python-sdk # Full platform skill (all 150+ apps via CLI) npx skills add inference-sh/skills@inference-sh # LLM models npx skills add inference-sh/skills@llm-models # Image generation npx skills add inference-sh/skills@ai-image-generation ``` ## Documentation - [JavaScript SDK Reference](https://inference.sh/docs/api/sdk-javascript) - Full API documentation - [Agent SDK Overview](https://inference.sh/docs/api/agent-sdk) - Building agents - [Tool Builder Reference](https://inference.sh/docs/api/agent-tools) - Creating tools - [Server Proxy Setup](https://inference.sh/docs/api/sdk/server-proxy) - Frontend integration - [Authentication](https://inference.sh/docs/api/authentication) - API key setup - [Streaming](https://inference.sh/docs/api/sdk/streaming) - Real-time updates - [File Uploads](https://inference.sh/docs/api/sdk/files) - File handling

path: references/agent-patterns.md
content: # Agent Patterns

Common patterns for building agents with the JavaScript SDK.

## Multi-Agent Orchestration

Delegate tasks to specialized sub-agents:

```typescript
import { inference, agentTool, string } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

// Define sub-agents as tools
const researcher = agentTool('research', 'my-org/researcher@latest')
  .describe('Research a topic thoroughly')
  .param('topic', string('Topic to research'))
  .build();

const writer = agentTool('write', 'my-org/writer@latest')
  .describe('Write content based on research')
  .param('outline', string('Content outline'))
  .param('research', string('Research findings'))
  .build();

// Orchestrator agent
const orchestrator = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: `You are an orchestrator that:
1. Uses the research tool to gather information
2. Uses the write tool to create content
Coordinate between agents to produce high-quality output.`,
  tools: [researcher, writer]
});

const response = await orchestrator.sendMessage('Create a blog post about AI agents');
```

## RAG Pattern (Retrieval-Augmented Generation)

Combine search with LLM responses:

```typescript
import { inference, appTool, string } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

// Search tool
const search = appTool('search', 'tavily/search-assistant@latest')
  .describe('Search the web for current information')
  .param('query', string('Search query'))
  .build();

// RAG agent
const ragAgent = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: `You help users with current information.
When asked about recent events or facts you're unsure about,
use the search tool to find accurate, up-to-date information.
Always cite your sources.`,
  tools: [search]
});

const response = await ragAgent.sendMessage(
  'What are the latest developments in quantum computing?'
);
```

## Code Execution Pattern

Agents that can write and run code:

```typescript
import { inference, internalTools } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

const config = internalTools()
  .codeExecution(true)
  .build();

const coder = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: `You are a coding assistant.
Write code to solve problems and execute it to verify it works.
Explain your approach and show the output.`,
  internal_tools: config
});

const response = await coder.sendMessage('Calculate the first 20 Fibonacci numbers');
```

## Human-in-the-Loop Pattern

Require approval for sensitive operations:

```typescript
import { inference, tool, string } from '@inferencesh/sdk';
import * as readline from 'readline';

const client = inference({ [REDACTED_SECRET]' });

// Tool requiring approval
const deleteFile = tool('delete_file')
  .describe('Delete a file from the filesystem')
  .param('path', string('File path to delete'))
  .requireApproval()
  .build();

async function promptUser(question: string): Promise<boolean> {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
  });

  return new Promise((resolve) => {
    rl.question(question, (answer) => {
      rl.close();
      resolve(answer.toLowerCase() === 'y');
    });
  });
}

const agent = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  tools: [deleteFile]
});

const response = await agent.sendMessage('Clean up temporary files in /tmp/myapp', {
  onToolCall: async (call) => {
    if (call.requiresApproval) {
      console.log(`\n⚠️  Agent wants to: ${call.name}`);
      console.log(`   Arguments: ${JSON.stringify(call.args)}`);
      const approved = await promptUser('Allow? (y/n): ');

      if (approved) {
        const result = await executeOperation(call.name, call.args);
        agent.submitToolResult(call.id, result);
      } else {
        agent.submitToolResult(call.id, { error: 'Operation denied by user' });
      }
    }
  }
});
```

## Conversation Memory Pattern

Maintain context across sessions:

```typescript
import { inference } from '@inferencesh/sdk';
import { readFileSync, writeFileSync, existsSync } from 'fs';

const client = inference({ [REDACTED_SECRET]' });

function saveChat(agent: any, filepath: string) {
  const chat = agent.getChat();
  writeFileSync(filepath, JSON.stringify(chat, null, 2));
}

async function loadAndContinue(filepath: string) {
  const agent = client.agent('my-org/assistant@latest');

  if (existsSync(filepath)) {
    const chat = JSON.parse(readFileSync(filepath, 'utf-8'));
    // Restore by replaying user messages
    for (const msg of chat.messages) {
      if (msg.role === 'user') {
        await agent.sendMessage(msg.content);
      }
    }
  }

  return agent;
}

// Usage
const agent = await loadAndContinue('conversation.json');
const response = await agent.sendMessage('Continue where we left off');
saveChat(agent, 'conversation.json');
```

## Streaming with Progress UI

Real-time updates for better UX:

```typescript
import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });
const agent = client.agent('my-org/assistant@latest');

const response = await agent.sendMessage('Generate a report on market trends', {
  onMessage: (msg) => {
    if (msg.content) {
      process.stdout.write(msg.content);
    }
  },
  onToolCall: async (call) => {
    console.log(`\n🔧 Using tool: ${call.name}`);
    const result = await executeTool(call.name, call.args);
    agent.submitToolResult(call.id, result);
    console.log('✅ Tool completed');
  }
});

console.log('\n\n📊 Report complete!');
```

## React Integration Pattern

Use in React components:

```typescript
import { useState, useCallback } from 'react';
import { inference } from '@inferencesh/sdk';

const client = inference({ proxyUrl: '/api/inference/proxy' });

function ChatComponent() {
  const [messages, setMessages] = useState<string[]>([]);
  const [input, setInput] = useState('');
  const [agent] = useState(() => client.agent('my-org/assistant@latest'));

  const sendMessage = useCallback(async () => {
    if (!input.trim()) return;

    setMessages(prev => [...prev, `You: ${input}`]);
    setInput('');

    let response = '';
    await agent.sendMessage(input, {
      onMessage: (msg) => {
        if (msg.content) {
          response += msg.content;
          setMessages(prev => [
            ...prev.slice(0, -1),
            `Assistant: ${response}`
          ]);
        }
      }
    });
  }, [input, agent]);

  return (
    <div>
      {messages.map((msg, i) => <div key={i}>{msg}</div>)}
      <input value={input} onChange={e => setInput(e.target.value)} />
      <button onClick={sendMessage}>Send</button>
    </div>
  );
}
```

## Error Recovery Pattern

Graceful handling of failures:

```typescript
import { inference, RequirementsNotMetException, InferenceError } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

async function robustRun(config: any, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await client.run(config);
    } catch (e) {
      if (e instanceof RequirementsNotMetException) {
        console.log('Missing requirements:', e.errors);
        throw e;
      }
      if (e instanceof InferenceError && attempt < maxRetries - 1) {
        const wait = Math.pow(2, attempt) * 1000;
        console.log(`Error: ${e.message}. Retrying in ${wait}ms...`);
        await new Promise(r => setTimeout(r, wait));
      } else {
        throw e;
      }
    }
  }
}

const result = await robustRun({
  app: 'infsh/flux-schnell',
  input: { prompt: 'A serene landscape' }
});
```

## Batch Processing Pattern

Process multiple items efficiently:

```typescript
import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

async function processBatch(items: string[], concurrency = 5) {
  const results: any[] = [];
  const queue = [...items];
  const inProgress: Promise<void>[] = [];

  async function processOne(item: string) {
    const result = await client.run({
      app: 'infsh/flux-schnell',
      input: { prompt: item }
    });
    results.push(result);
  }

  while (queue.length > 0 || inProgress.length > 0) {
    // Fill up to concurrency limit
    while (queue.length > 0 && inProgress.length < concurrency) {
      const item = queue.shift()!;
      const promise = processOne(item).then(() => {
        const index = inProgress.indexOf(promise);
        if (index > -1) inProgress.splice(index, 1);
      });
      inProgress.push(promise);
    }

    // Wait for at least one to complete
    if (inProgress.length > 0) {
      await Promise.race(inProgress);
    }
  }

  return results;
}

const prompts = [
  'A mountain sunrise',
  'A city at night',
  'An ocean sunset',
  'A forest path'
];

const results = await processBatch(prompts);
```

## Next.js Server Actions Pattern

Use with React Server Components:

```typescript
// app/actions.ts
'use server';

import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET] });

export async function generateImage(prompt: string) {
  const result = await client.run({
    app: 'infsh/flux-schnell',
    input: { prompt }
  });
  return result.output;
}

export async function chat(message: string, sessionId?: string) {
  const agent = client.agent('my-org/assistant@latest');
  const response = await agent.sendMessage(message);
  return {
    text: response.text,
    sessionId: response.sessionId
  };
}
```

```typescript
// app/page.tsx
import { generateImage, chat } from './actions';

export default function Page() {
  async function handleSubmit(formData: FormData) {
    'use server';
    const prompt = formData.get('prompt') as string;
    const image = await generateImage(prompt);
    // Handle result
  }

  return (
    <form action={handleSubmit}>
      <input name="prompt" placeholder="Describe an image..." />
      <button type="submit">Generate</button>
    </form>
  );
}
```


path: references/files.md
content: # File Handling Reference

Upload, download, and manage files with the JavaScript SDK.

## Automatic File Upload

Local file paths in input are automatically uploaded (Node.js):

```typescript
import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

// File path is auto-uploaded
const result = await client.run({
  app: 'image-processor',
  input: {
    image: '/path/to/image.png'
  }
});
```

## Manual File Upload

### Node.js

```typescript
// Basic upload
const file = await client.uploadFile('/path/to/image.png');
console.log(file.uri); // inf://files/abc123

const result = await client.run({
  app: 'image-processor',
  input: { image: file.uri }
});
```

### Upload Options

```typescript
const file = await client.uploadFile('/path/to/document.pdf', {
  filename: 'custom_name.pdf',      // Custom filename
  contentType: 'application/pdf',   // MIME type
  path: '/documents/reports',        // Storage path
  public: true                       // Publicly accessible
});
```

## Browser File Upload

### From File Input

```typescript
const input = document.querySelector<HTMLInputElement>('input[type="file"]');

input.addEventListener('change', async (e) => {
  const file = input.files?.[0];
  if (!file) return;

  const uploaded = await client.uploadFile(file);
  console.log('Uploaded:', uploaded.uri);
});
```

### With React

```typescript
import { useState } from 'react';
import { inference } from '@inferencesh/sdk';

function FileUploader() {
  const [uploading, setUploading] = useState(false);
  const client = inference({ proxyUrl: '/api/inference/proxy' });

  async function handleFile(e: React.ChangeEvent<HTMLInputElement>) {
    const file = e.target.files?.[0];
    if (!file) return;

    setUploading(true);
    try {
      const uploaded = await client.uploadFile(file);
      console.log('Uploaded:', uploaded.uri);
    } finally {
      setUploading(false);
    }
  }

  return (
    <div>
      <input type="file" onChange={handleFile} disabled={uploading} />
      {uploading && <span>Uploading...</span>}
    </div>
  );
}
```

### Drag and Drop

```typescript
function DropZone() {
  const client = inference({ proxyUrl: '/api/inference/proxy' });

  async function handleDrop(e: React.DragEvent) {
    e.preventDefault();
    const files = Array.from(e.dataTransfer.files);

    for (const file of files) {
      const uploaded = await client.uploadFile(file);
      console.log(`Uploaded ${file.name}:`, uploaded.uri);
    }
  }

  return (
    <div
      onDrop={handleDrop}
      onDragOver={(e) => e.preventDefault()}
      style={{ border: '2px dashed #ccc', padding: 20 }}
    >
      Drop files here
    </div>
  );
}
```

## Supported Input Types

### File Path (Node.js)

```typescript
const result = await client.run({
  app: 'processor',
  input: { file: '/path/to/file.png' }
});
```

### Data URI (Base64)

```typescript
import { readFileSync } from 'fs';

const buffer = readFileSync('image.png');
const b64 = buffer.toString('base64');

const result = await client.run({
  app: 'processor',
  input: { image: `data:image/png;base64,${b64}` }
});
```

### Buffer (Node.js)

```typescript
import { readFileSync } from 'fs';

const buffer = readFileSync('image.png');

const file = await client.uploadFile(buffer, {
  filename: 'image.png',
  contentType: 'image/png'
});
```

### Blob (Browser)

```typescript
// From canvas
const canvas = document.querySelector('canvas');
canvas.toBlob(async (blob) => {
  if (!blob) return;

  const file = await client.uploadFile(blob, {
    filename: 'canvas.png',
    contentType: 'image/png'
  });
  console.log('Uploaded:', file.uri);
});
```

### File Object (Browser)

```typescript
const fileInput = document.querySelector<HTMLInputElement>('input[type="file"]');
const file = fileInput.files?.[0];

if (file) {
  const uploaded = await client.uploadFile(file);
}
```

## Working with URLs

Use remote URLs directly (no upload needed):

```typescript
const result = await client.run({
  app: 'image-processor',
  input: {
    image: 'https://example.com/image.png'
  }
});
```

## Multiple Files

```typescript
// Upload multiple files
const files = await Promise.all([
  client.uploadFile('/path/to/file1.png'),
  client.uploadFile('/path/to/file2.png')
]);

const result = await client.run({
  app: 'multi-file-processor',
  input: { images: files.map(f => f.uri) }
});
```

## File Info

```typescript
const file = await client.uploadFile('/path/to/image.png');

console.log({
  uri: file.uri,        // inf://files/abc123
  url: file.url,        // Direct access URL
  size: file.size,      // File size in bytes
  contentType: file.contentType
});
```

## Downloading Results

### Node.js

```typescript
import { writeFileSync } from 'fs';

const result = await client.run({
  app: 'infsh/flux-schnell',
  input: { prompt: 'A sunset' }
});

// Result contains URL to generated file
const imageUrl = result.output.image;

// Download the file
const response = await fetch(imageUrl);
const buffer = Buffer.from(await response.arrayBuffer());
writeFileSync('output.png', buffer);
```

### Browser

```typescript
const result = await client.run({
  app: 'infsh/flux-schnell',
  input: { prompt: 'A sunset' }
});

// Trigger download
const link = document.createElement('a');
link.href = result.output.image;
link.download = 'generated.png';
link.click();
```

## Agent File Attachments

### Node.js

```typescript
import { readFileSync } from 'fs';

const agent = client.agent('my-org/assistant@latest');

// From buffer
const response = await agent.sendMessage('What\'s in this image?', {
  files: [readFileSync('image.png')]
});
```

### Browser

```typescript
const agent = client.agent('my-org/assistant@latest');

// From file input
const input = document.querySelector<HTMLInputElement>('input[type="file"]');
const file = input.files?.[0];

if (file) {
  const response = await agent.sendMessage('Describe this image', {
    files: [file]
  });
}
```

### From Base64

```typescript
const response = await agent.sendMessage('Analyze this document', {
  files: ['data:application/pdf;base64,JVBERi0xLj...']
});
```

### Multiple Files

```typescript
const response = await agent.sendMessage('Compare these images', {
  files: [
    await fetch('image1.png').then(r => r.blob()),
    await fetch('image2.png').then(r => r.blob())
  ]
});
```

## Content Type Detection

```typescript
function getContentType(filename: string): string {
  const ext = filename.split('.').pop()?.toLowerCase();
  const types: Record<string, string> = {
    png: 'image/png',
    jpg: 'image/jpeg',
    jpeg: 'image/jpeg',
    gif: 'image/gif',
    webp: 'image/webp',
    pdf: 'application/pdf',
    mp4: 'video/mp4',
    mp3: 'audio/mpeg',
    wav: 'audio/wav'
  };
  return types[ext || ''] || 'application/octet-stream';
}

async function uploadWithAutoType(path: string) {
  const filename = path.split('/').pop() || 'file';
  return client.uploadFile(path, {
    contentType: getContentType(filename)
  });
}
```

## React Hook for File Upload

```typescript
import { useState, useCallback } from 'react';
import { inference } from '@inferencesh/sdk';

interface UploadState {
  uploading: boolean;
  progress: number;
  error: string | null;
  file: any | null;
}

function useFileUpload() {
  const [state, setState] = useState<UploadState>({
    uploading: false,
    progress: 0,
    error: null,
    file: null
  });

  const client = inference({ proxyUrl: '/api/inference/proxy' });

  const upload = useCallback(async (file: File) => {
    setState({ uploading: true, progress: 0, error: null, file: null });

    try {
      const uploaded = await client.uploadFile(file, {
        onProgress: (progress) => {
          setState(prev => ({ ...prev, progress }));
        }
      });

      setState({ uploading: false, progress: 100, error: null, file: uploaded });
      return uploaded;
    } catch (e: any) {
      setState(prev => ({ ...prev, uploading: false, error: e.message }));
      throw e;
    }
  }, []);

  return { ...state, upload };
}
```

## Error Handling

```typescript
import { FileUploadError } from '@inferencesh/sdk';

try {
  const file = await client.uploadFile(largeFile);
} catch (e) {
  if (e instanceof FileUploadError) {
    if (e.message.includes('too large')) {
      console.log('File exceeds size limit');
    } else if (e.message.includes('unsupported')) {
      console.log('File type not supported');
    } else {
      console.log('Upload failed:', e.message);
    }
  }
}
```

## Stream Upload (Large Files)

```typescript
import { createReadStream } from 'fs';
import { stat } from 'fs/promises';

async function uploadLargeFile(filepath: string) {
  const stats = await stat(filepath);
  const stream = createReadStream(filepath);

  const file = await client.uploadFile(stream, {
    filename: filepath.split('/').pop(),
    contentType: 'application/octet-stream',
    size: stats.size
  });

  return file;
}
```


path: references/react-integration.md
content: # React Integration Reference

Build AI-powered React applications with the inference.sh SDK.

## Setup

```bash
npm install @inferencesh/sdk
```

Configure proxy (keep API keys on server):

```typescript
// app/api/inference/proxy/route.ts (Next.js)
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export const POST = route.POST;
```

## Basic Client Setup

```typescript
import { inference } from '@inferencesh/sdk';

// Create client with proxy (no API key in browser)
const client = inference({ proxyUrl: '/api/inference/proxy' });
```

## useInference Hook

Custom hook for running AI apps:

```typescript
import { useState, useCallback } from 'react';
import { inference } from '@inferencesh/sdk';

interface UseInferenceState<T> {
  data: T | null;
  loading: boolean;
  error: string | null;
}

function useInference<T = any>() {
  const [state, setState] = useState<UseInferenceState<T>>({
    data: null,
    loading: false,
    error: null
  });

  const client = inference({ proxyUrl: '/api/inference/proxy' });

  const run = useCallback(async (config: any) => {
    setState({ data: null, loading: true, error: null });

    try {
      const result = await client.run(config);
      setState({ data: result.output as T, loading: false, error: null });
      return result.output as T;
    } catch (e: any) {
      setState({ data: null, loading: false, error: e.message });
      throw e;
    }
  }, []);

  return { ...state, run };
}

// Usage
function ImageGenerator() {
  const { data, loading, error, run } = useInference<{ image: string }>();

  const generate = () => run({
    app: 'infsh/flux-schnell',
    input: { prompt: 'A sunset' }
  });

  return (
    <div>
      <button onClick={generate} disabled={loading}>
        {loading ? 'Generating...' : 'Generate'}
      </button>
      {error && <div className="error">{error}</div>}
      {data && <img src={data.image} alt="Generated" />}
    </div>
  );
}
```

## useChat Hook

Conversational AI with streaming:

```typescript
import { useState, useCallback, useRef } from 'react';
import { inference } from '@inferencesh/sdk';

interface Message {
  role: 'user' | 'assistant';
  content: string;
}

function useChat(agentRef: string) {
  const [messages, setMessages] = useState<Message[]>([]);
  const [loading, setLoading] = useState(false);
  const [streaming, setStreaming] = useState(false);

  const clientRef = useRef(inference({ proxyUrl: '/api/inference/proxy' }));
  const agentRef_ = useRef(clientRef.current.agent(agentRef));

  const sendMessage = useCallback(async (content: string) => {
    // Add user message
    setMessages(prev => [...prev, { role: 'user', content }]);
    setLoading(true);
    setStreaming(true);

    // Placeholder for assistant response
    let assistantContent = '';
    setMessages(prev => [...prev, { role: 'assistant', content: '' }]);

    try {
      await agentRef_.current.sendMessage(content, {
        onMessage: (msg) => {
          if (msg.content) {
            assistantContent += msg.content;
            setMessages(prev => [
              ...prev.slice(0, -1),
              { role: 'assistant', content: assistantContent }
            ]);
          }
        }
      });
    } finally {
      setLoading(false);
      setStreaming(false);
    }
  }, []);

  const reset = useCallback(() => {
    agentRef_.current.reset();
    setMessages([]);
  }, []);

  return { messages, loading, streaming, sendMessage, reset };
}

// Usage
function ChatComponent() {
  const { messages, loading, sendMessage, reset } = useChat('my-org/assistant@latest');
  const [input, setInput] = useState('');

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (input.trim()) {
      sendMessage(input);
      setInput('');
    }
  };

  return (
    <div className="chat">
      <div className="messages">
        {messages.map((msg, i) => (
          <div key={i} className={`message ${msg.role}`}>
            {msg.content}
          </div>
        ))}
      </div>

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Type a message..."
          disabled={loading}
        />
        <button type="submit" disabled={loading}>Send</button>
      </form>

      <button onClick={reset}>New Chat</button>
    </div>
  );
}
```

## useStream Hook

Stream task progress:

```typescript
import { useState, useCallback, useRef } from 'react';
import { inference } from '@inferencesh/sdk';

interface StreamState {
  status: string;
  progress: number;
  logs: string[];
  output: any;
  error: string | null;
}

function useStream() {
  const [state, setState] = useState<StreamState>({
    status: 'idle',
    progress: 0,
    logs: [],
    output: null,
    error: null
  });

  const controllerRef = useRef<AbortController | null>(null);
  const client = inference({ proxyUrl: '/api/inference/proxy' });

  const run = useCallback(async (config: any) => {
    controllerRef.current = new AbortController();

    setState({
      status: 'starting',
      progress: 0,
      logs: [],
      output: null,
      error: null
    });

    try {
      const stream = await client.run(config, {
        stream: true,
        signal: controllerRef.current.signal
      });

      for await (const update of stream) {
        setState(prev => ({
          ...prev,
          status: update.status,
          progress: update.progress
            ? (update.progress.current / update.progress.total) * 100
            : prev.progress,
          logs: update.logs ? [...prev.logs, ...update.logs] : prev.logs,
          output: update.output || prev.output
        }));
      }
    } catch (e: any) {
      if (e.name !== 'AbortError') {
        setState(prev => ({ ...prev, status: 'error', error: e.message }));
      }
    }
  }, []);

  const cancel = useCallback(() => {
    controllerRef.current?.abort();
    setState(prev => ({ ...prev, status: 'cancelled' }));
  }, []);

  return { ...state, run, cancel };
}

// Usage
function VideoGenerator() {
  const { status, progress, logs, output, run, cancel } = useStream();

  return (
    <div>
      <button onClick={() => run({
        app: 'google/veo-3-1-fast',
        input: { prompt: 'Ocean waves' }
      })}>
        Generate Video
      </button>
      <button onClick={cancel}>Cancel</button>

      <div>Status: {status}</div>
      <progress value={progress} max={100} />

      <div className="logs">
        {logs.map((log, i) => <div key={i}>{log}</div>)}
      </div>

      {output && <video src={output.video} controls />}
    </div>
  );
}
```

## File Upload Component

```typescript
import { useState, useCallback } from 'react';
import { inference } from '@inferencesh/sdk';

function FileUpload({ onUpload }: { onUpload: (uri: string) => void }) {
  const [uploading, setUploading] = useState(false);
  const [preview, setPreview] = useState<string | null>(null);

  const client = inference({ proxyUrl: '/api/inference/proxy' });

  const handleFile = useCallback(async (e: React.ChangeEvent<HTMLInputElement>) => {
    const file = e.target.files?.[0];
    if (!file) return;

    // Show preview
    const reader = new FileReader();
    reader.onload = () => setPreview(reader.result as string);
    reader.readAsDataURL(file);

    // Upload
    setUploading(true);
    try {
      const uploaded = await client.uploadFile(file);
      onUpload(uploaded.uri);
    } finally {
      setUploading(false);
    }
  }, [onUpload]);

  return (
    <div>
      <input type="file" onChange={handleFile} disabled={uploading} />
      {uploading && <span>Uploading...</span>}
      {preview && <img src={preview} alt="Preview" style={{ maxWidth: 200 }} />}
    </div>
  );
}
```

## Image Analysis Component

```typescript
function ImageAnalyzer() {
  const [imageUri, setImageUri] = useState<string | null>(null);
  const { messages, sendMessage, loading } = useChat('my-org/vision-assistant@latest');

  const handleUpload = (uri: string) => {
    setImageUri(uri);
  };

  const analyze = () => {
    if (imageUri) {
      sendMessage(`Analyze this image: ${imageUri}`);
    }
  };

  return (
    <div>
      <FileUpload onUpload={handleUpload} />

      <button onClick={analyze} disabled={!imageUri || loading}>
        Analyze Image
      </button>

      <div className="messages">
        {messages.map((msg, i) => (
          <div key={i} className={msg.role}>{msg.content}</div>
        ))}
      </div>
    </div>
  );
}
```

## Tool Handling Component

```typescript
import { useState, useCallback } from 'react';
import { inference, tool, string } from '@inferencesh/sdk';

function AgentWithTools() {
  const [response, setResponse] = useState('');
  const [pendingTool, setPendingTool] = useState<any>(null);

  const client = inference({ proxyUrl: '/api/inference/proxy' });

  const confirmTool = tool('confirm_action')
    .describe('Confirm a destructive action')
    .param('action', string('Action to confirm'))
    .requireApproval()
    .build();

  const agent = client.agent({
    core_app: { ref: 'infsh/claude-sonnet-4@latest' },
    tools: [confirmTool]
  });

  const handleMessage = useCallback(async (message: string) => {
    setResponse('');

    await agent.sendMessage(message, {
      onMessage: (msg) => {
        if (msg.content) {
          setResponse(prev => prev + msg.content);
        }
      },
      onToolCall: (call) => {
        if (call.requiresApproval) {
          setPendingTool(call);
        }
      }
    });
  }, []);

  const handleApprove = (approved: boolean) => {
    if (pendingTool) {
      if (approved) {
        agent.submitToolResult(pendingTool.id, { confirmed: true });
      } else {
        agent.submitToolResult(pendingTool.id, { error: 'Denied by user' });
      }
      setPendingTool(null);
    }
  };

  return (
    <div>
      <button onClick={() => handleMessage('Delete all temp files')}>
        Run Agent
      </button>

      <div>{response}</div>

      {pendingTool && (
        <div className="approval-dialog">
          <p>Agent wants to: {pendingTool.args.action}</p>
          <button onClick={() => handleApprove(true)}>Approve</button>
          <button onClick={() => handleApprove(false)}>Deny</button>
        </div>
      )}
    </div>
  );
}
```

## Context Provider

Share client across components:

```typescript
import { createContext, useContext, useMemo, ReactNode } from 'react';
import { inference, type InferenceClient } from '@inferencesh/sdk';

const InferenceContext = createContext<InferenceClient | null>(null);

export function InferenceProvider({ children }: { children: ReactNode }) {
  const client = useMemo(() =>
    inference({ proxyUrl: '/api/inference/proxy' }),
    []
  );

  return (
    <InferenceContext.Provider value={client}>
      {children}
    </InferenceContext.Provider>
  );
}

export function useInferenceClient() {
  const client = useContext(InferenceContext);
  if (!client) {
    throw new Error('useInferenceClient must be used within InferenceProvider');
  }
  return client;
}

// Usage in app
function App() {
  return (
    <InferenceProvider>
      <ChatComponent />
      <ImageGenerator />
    </InferenceProvider>
  );
}

// Usage in component
function MyComponent() {
  const client = useInferenceClient();
  // Use client...
}
```

## Suspense Integration

```typescript
import { Suspense } from 'react';
import { use } from 'react';
import { inference } from '@inferencesh/sdk';

const client = inference({ proxyUrl: '/api/inference/proxy' });

// Create a promise for the resource
function createImageResource(prompt: string) {
  const promise = client.run({
    app: 'infsh/flux-schnell',
    input: { prompt }
  }).then(r => r.output.image);

  return { read: () => use(promise) };
}

function GeneratedImage({ resource }: { resource: ReturnType<typeof createImageResource> }) {
  const imageUrl = resource.read();
  return <img src={imageUrl} alt="Generated" />;
}

function App() {
  const [resource, setResource] = useState<any>(null);

  const generate = () => {
    setResource(createImageResource('A sunset'));
  };

  return (
    <div>
      <button onClick={generate}>Generate</button>
      {resource && (
        <Suspense fallback={<div>Generating...</div>}>
          <GeneratedImage resource={resource} />
        </Suspense>
      )}
    </div>
  );
}
```

## Error Boundary

```typescript
import { Component, ReactNode } from 'react';

interface Props {
  children: ReactNode;
  fallback?: ReactNode;
}

interface State {
  hasError: boolean;
  error: Error | null;
}

class InferenceErrorBoundary extends Component<Props, State> {
  state: State = { hasError: false, error: null };

  static getDerivedStateFromError(error: Error): State {
    return { hasError: true, error };
  }

  render() {
    if (this.state.hasError) {
      return this.props.fallback || (
        <div className="error">
          <h2>Something went wrong</h2>
          <p>{this.state.error?.message}</p>
          <button onClick={() => this.setState({ hasError: false, error: null })}>
            Try again
          </button>
        </div>
      );
    }

    return this.props.children;
  }
}

// Usage
function App() {
  return (
    <InferenceErrorBoundary>
      <AIFeatures />
    </InferenceErrorBoundary>
  );
}
```


path: references/server-proxy.md
content: # Server Proxy Setup

Secure API key handling for frontend applications.

## Why Use a Proxy?

Never expose API keys in browser code. Use a server proxy to:

- Keep API keys secure on the server
- Add authentication/authorization
- Rate limit requests
- Log usage and analytics

```
Browser (SDK) → Your Backend (Proxy) → api.inference.sh
  (no key)     (injects API key)      (authenticated)
```

## Client Configuration

```typescript
import { inference } from '@inferencesh/sdk';

// Frontend code - no API key!
const client = inference({
  proxyUrl: '/api/inference/proxy'
});

// Use normally
const result = await client.run({
  app: 'infsh/flux-schnell',
  input: { prompt: 'A sunset' }
});
```

## Next.js (App Router)

```typescript
// app/api/inference/proxy/route.ts
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export const POST = route.POST;
```

### With Custom Authentication

```typescript
// app/api/inference/proxy/route.ts
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';
import { getServerSession } from 'next-auth';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export async function POST(req: Request) {
  // Check auth first
  const session = await getServerSession();
  if (!session) {
    return Response.json({ error: 'Unauthorized' }, { status: 401 });
  }

  // Check rate limits, log usage, etc.

  return route.POST(req);
}
```

## Next.js (Pages Router)

```typescript
// pages/api/inference/proxy.ts
import { createApiHandler } from '@inferencesh/sdk/proxy/nextjs-pages';

export default createApiHandler({
  [REDACTED_SECRET]
});
```

## Express

```typescript
import express from 'express';
import { createProxyMiddleware } from '@inferencesh/sdk/proxy/express';

const app = express();

// Basic setup
app.use('/api/inference/proxy', createProxyMiddleware({
  [REDACTED_SECRET]
}));

// With auth middleware
app.use('/api/inference/proxy',
  requireAuth,
  createProxyMiddleware({
    [REDACTED_SECRET]
  })
);

app.listen(3000);
```

## Hono

```typescript
import { Hono } from 'hono';
import { createMiddleware } from '@inferencesh/sdk/proxy/hono';

const app = new Hono();

app.post('/api/inference/proxy', createMiddleware({
  [REDACTED_SECRET]
}));

export default app;
```

### Cloudflare Workers

```typescript
import { Hono } from 'hono';
import { createMiddleware } from '@inferencesh/sdk/proxy/hono';

type Bindings = {
  INFERENCE_API_KEY: string;
};

const app = new Hono<{ Bindings: Bindings }>();

app.post('/api/inference/proxy', (c) => {
  const middleware = createMiddleware({
    [REDACTED_SECRET]
  });
  return middleware(c);
});

export default app;
```

## Remix

```typescript
// app/routes/api.inference.proxy.tsx
import type { ActionFunctionArgs } from '@remix-run/node';
import { createActionHandler } from '@inferencesh/sdk/proxy/remix';

const handler = createActionHandler({
  [REDACTED_SECRET]
});

export const action = async ({ request }: ActionFunctionArgs) => {
  return handler(request);
};
```

## SvelteKit

```typescript
// src/routes/api/inference/proxy/+server.ts
import { createRequestHandler } from '@inferencesh/sdk/proxy/sveltekit';
import { INFERENCE_API_KEY } from '$env/static/private';

const handler = createRequestHandler({
  [REDACTED_SECRET]
});

export const POST = handler;
```

## Dynamic API Key Resolution

Load API keys from secrets managers:

```typescript
// Next.js example
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';
import { getSecret } from './vault';

const route = createRouteHandler({
  resolveApiKey: async () => {
    return await getSecret('INFERENCE_API_KEY');
  }
});

export const POST = route.POST;
```

## Per-User API Keys

Use different keys per user/organization:

```typescript
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';
import { getServerSession } from 'next-auth';
import { db } from './db';

const route = createRouteHandler({
  resolveApiKey: async (req) => {
    const session = await getServerSession();
    if (!session?.user) {
      throw new Error('Unauthorized');
    }

    // Get user's API key from database
    const user = await db.user.findUnique({
      where: { id: session.user.id },
      select: { inferenceApiKey: true }
    });

    return user?.inferenceApiKey || process.env.DEFAULT_INFERENCE_API_KEY!;
  }
});

export const POST = route.POST;
```

## Rate Limiting

```typescript
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';
import { Ratelimit } from '@upstash/ratelimit';
import { Redis } from '@upstash/redis';

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, '1 m'), // 10 requests per minute
});

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export async function POST(req: Request) {
  // Get user identifier
  const ip = req.headers.get('x-forwarded-for') || 'anonymous';

  // Check rate limit
  const { success, limit, reset, remaining } = await ratelimit.limit(ip);

  if (!success) {
    return Response.json(
      { error: 'Rate limit exceeded' },
      {
        status: 429,
        headers: {
          'X-RateLimit-Limit': limit.toString(),
          'X-RateLimit-Remaining': remaining.toString(),
          'X-RateLimit-Reset': reset.toString()
        }
      }
    );
  }

  return route.POST(req);
}
```

## Usage Logging

```typescript
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';
import { getServerSession } from 'next-auth';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export async function POST(req: Request) {
  const session = await getServerSession();
  const body = await req.clone().json();

  // Log before
  console.log({
    type: 'inference_request',
    userId: session?.user?.id,
    app: body.app,
    timestamp: new Date().toISOString()
  });

  const response = await route.POST(req);

  // Log after
  console.log({
    type: 'inference_response',
    userId: session?.user?.id,
    status: response.status,
    timestamp: new Date().toISOString()
  });

  return response;
}
```

## Error Handling

```typescript
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export async function POST(req: Request) {
  try {
    return await route.POST(req);
  } catch (error) {
    console.error('Proxy error:', error);

    // Don't expose internal errors to client
    return Response.json(
      { error: 'Internal server error' },
      { status: 500 }
    );
  }
}
```

## Vercel Deployment

```typescript
// vercel.json
{
  "functions": {
    "app/api/inference/proxy/route.ts": {
      "maxDuration": 60
    }
  }
}
```

```typescript
// app/api/inference/proxy/route.ts
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export const POST = route.POST;

// Enable streaming for long-running requests
export const runtime = 'edge';
```

## CORS Configuration

If your frontend and backend are on different domains:

```typescript
import { createRouteHandler } from '@inferencesh/sdk/proxy/nextjs';

const route = createRouteHandler({
  [REDACTED_SECRET]
});

export async function POST(req: Request) {
  const response = await route.POST(req);

  // Add CORS headers
  const headers = new Headers(response.headers);
  headers.set('Access-Control-Allow-Origin', 'https://your-frontend.com');
  headers.set('Access-Control-Allow-Methods', 'POST');
  headers.set('Access-Control-Allow-Headers', 'Content-Type');

  return new Response(response.body, {
    status: response.status,
    headers
  });
}

export async function OPTIONS() {
  return new Response(null, {
    headers: {
      'Access-Control-Allow-Origin': 'https://your-frontend.com',
      'Access-Control-Allow-Methods': 'POST, OPTIONS',
      'Access-Control-Allow-Headers': 'Content-Type'
    }
  });
}
```

## Testing Locally

```typescript
// Use environment variable for flexibility
const client = inference({
  // Use proxy in production, direct API in development
  ...(process.env.NODE_ENV === 'production'
    ? { proxyUrl: '/api/inference/proxy' }
    : { [REDACTED_SECRET] })
});
```


path: references/sessions.md
content: # Sessions Reference

Stateful execution with warm workers.

## What Are Sessions?

Sessions keep workers warm between requests, enabling:

- **Faster execution** - No cold start on subsequent calls
- **Shared state** - Maintain context, loaded models, cached data
- **Cost efficiency** - Reuse initialized resources

## Creating a Session

```typescript
import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

// Start new session
const result = await client.run({
  app: 'my-app',
  input: { action: 'initialize' },
  session: 'new'
});

const sessionId = result.session_id;
console.log(`Session: ${sessionId}`);
```

## Using an Existing Session

```typescript
// Continue in same session
const result = await client.run({
  app: 'my-app',
  input: { action: 'process', data: '...' },
  session: sessionId
});
```

## Session Timeout

Set how long idle sessions stay alive (1-3600 seconds):

```typescript
// 5-minute timeout
const result = await client.run({
  app: 'my-app',
  input: { action: 'init' },
  session: 'new',
  session_timeout: 300
});
```

## Session Lifecycle

```
1. Create session (session: "new")
   ↓
2. Worker starts, initializes app
   ↓
3. Subsequent calls reuse worker (session: sessionId)
   ↓
4. Idle timeout reached or explicit close
   ↓
5. Worker terminates
```

## Use Cases

### Model Loading

Load a model once, use it multiple times:

```typescript
// Initial load (slow)
const result = await client.run({
  app: 'ml-inference',
  input: { action: 'load_model', model: 'large-model-v2' },
  session: 'new',
  session_timeout: 600
});
const sessionId = result.session_id;

// Fast inference calls
for (const item of dataBatch) {
  const result = await client.run({
    app: 'ml-inference',
    input: { action: 'predict', data: item },
    session: sessionId
  });
  console.log(result.output);
}
```

### Browser Automation

Keep browser open across multiple actions:

```typescript
// Start browser session
const result = await client.run({
  app: 'browser-automation',
  input: { action: 'start', url: 'https://example.com' },
  session: 'new',
  session_timeout: 300
});
const sessionId = result.session_id;

// Navigate
await client.run({
  app: 'browser-automation',
  input: { action: 'click', selector: '#login-btn' },
  session: sessionId
});

// Fill form
await client.run({
  app: 'browser-automation',
  input: { action: 'type', selector: '#username', text: '[REDACTED_SECRET]' },
  session: sessionId
});

// Take screenshot
const screenshot = await client.run({
  app: 'browser-automation',
  input: { action: 'screenshot' },
  session: sessionId
});
```

### Stateful Conversations

```typescript
// Initialize chat context
const result = await client.run({
  app: 'chat-with-memory',
  input: { action: 'init', system: 'You are a helpful assistant.' },
  session: 'new',
  session_timeout: 1800  // 30 minutes
});
const sessionId = result.session_id;

// Multi-turn conversation
const messages = [
  'What is quantum computing?',
  'Can you give me a simple example?',
  'How is it different from classical computing?'
];

for (const msg of messages) {
  const result = await client.run({
    app: 'chat-with-memory',
    input: { message: msg },
    session: sessionId
  });
  console.log(`Assistant: ${result.output.response}`);
}
```

### Data Processing Pipeline

```typescript
// Load data once
const result = await client.run({
  app: 'data-processor',
  input: { action: 'load', dataset: 'large_dataset.parquet' },
  session: 'new',
  session_timeout: 900
});
const sessionId = result.session_id;

// Run multiple analyses
const analyses = ['summary', 'correlations', 'outliers', 'trends'];

for (const analysis of analyses) {
  const result = await client.run({
    app: 'data-processor',
    input: { action: 'analyze', type: analysis },
    session: sessionId
  });
  console.log(`${analysis}:`, result.output);
}
```

## Session Management

### Session Recovery Pattern

```typescript
class SessionManager {
  private client: any;
  private app: string;
  private timeout: number;
  private sessionId: string | null = null;

  constructor(client: any, app: string, timeout = 300) {
    this.client = client;
    this.app = app;
    this.timeout = timeout;
  }

  private async ensureSession(): Promise<string> {
    if (!this.sessionId) {
      const result = await this.client.run({
        app: this.app,
        input: { action: 'init' },
        session: 'new',
        session_timeout: this.timeout
      });
      this.sessionId = result.session_id;
    }
    return this.sessionId;
  }

  async run(input: any) {
    try {
      return await this.client.run({
        app: this.app,
        input,
        session: await this.ensureSession()
      });
    } catch (e: any) {
      if (e.message?.toLowerCase().includes('session')) {
        // Session expired, create new one
        this.sessionId = null;
        return await this.client.run({
          app: this.app,
          input,
          session: await this.ensureSession()
        });
      }
      throw e;
    }
  }
}

// Usage
const manager = new SessionManager(client, 'my-app', 600);
const result = await manager.run({ action: 'process', data: '...' });
```

## React Hook for Sessions

```typescript
import { useState, useCallback, useRef } from 'react';
import { inference } from '@inferencesh/sdk';

function useSession(app: string, timeout = 300) {
  const [sessionId, setSessionId] = useState<string | null>(null);
  const [loading, setLoading] = useState(false);
  const clientRef = useRef(inference({ proxyUrl: '/api/inference/proxy' }));

  const run = useCallback(async (input: any) => {
    setLoading(true);

    try {
      const isNew = !sessionId;
      const result = await clientRef.current.run({
        app,
        input,
        session: isNew ? 'new' : sessionId,
        ...(isNew && { session_timeout: timeout })
      });

      if (isNew) {
        setSessionId(result.session_id);
      }

      return result;
    } catch (e: any) {
      if (e.message?.toLowerCase().includes('session')) {
        // Session expired
        setSessionId(null);
      }
      throw e;
    } finally {
      setLoading(false);
    }
  }, [app, sessionId, timeout]);

  const reset = useCallback(() => {
    setSessionId(null);
  }, []);

  return { sessionId, loading, run, reset };
}

// Usage
function BrowserAutomation() {
  const { sessionId, loading, run, reset } = useSession('browser-automation');

  return (
    <div>
      <div>Session: {sessionId || 'None'}</div>
      <button onClick={() => run({ action: 'start', url: '...' })} disabled={loading}>
        Start Browser
      </button>
      <button onClick={() => run({ action: 'screenshot' })} disabled={!sessionId || loading}>
        Screenshot
      </button>
      <button onClick={reset}>End Session</button>
    </div>
  );
}
```

## Express Session API

```typescript
import express from 'express';
import { inference } from '@inferencesh/sdk';

const app = express();
const client = inference({ [REDACTED_SECRET] });

// Store sessions per user
const userSessions: Map<string, string> = new Map();

app.post('/api/browser/start', async (req, res) => {
  const userId = req.user.id;

  const result = await client.run({
    app: 'browser-automation',
    input: { action: 'start', url: req.body.url },
    session: 'new',
    session_timeout: 300
  });

  userSessions.set(userId, result.session_id);
  res.json({ sessionId: result.session_id });
});

app.post('/api/browser/action', async (req, res) => {
  const userId = req.user.id;
  const sessionId = userSessions.get(userId);

  if (!sessionId) {
    return res.status(400).json({ error: 'No active session' });
  }

  try {
    const result = await client.run({
      app: 'browser-automation',
      input: req.body,
      session: sessionId
    });
    res.json(result.output);
  } catch (e: any) {
    if (e.message?.includes('session')) {
      userSessions.delete(userId);
      res.status(400).json({ error: 'Session expired' });
    } else {
      throw e;
    }
  }
});

app.post('/api/browser/end', (req, res) => {
  const userId = req.user.id;
  userSessions.delete(userId);
  res.json({ ok: true });
});
```

## Best Practices

1. **Set appropriate timeouts** - Balance between keeping workers warm and resource usage
2. **Handle session expiry** - Always catch and handle session not found errors
3. **Clean up when done** - Delete session references when user is finished
4. **Don't over-parallelize** - Session requests go to the same worker sequentially
5. **Monitor costs** - Long-running sessions incur ongoing charges
6. **Store session IDs securely** - Don't expose session IDs to untrusted clients


path: references/streaming.md
content: # Streaming Reference

Real-time progress updates and Server-Sent Events (SSE) handling.

## Task Status Flow

```
RECEIVED (1) → QUEUED (2) → SCHEDULED (3) → PREPARING (4)
→ SERVING (5) → SETTING_UP (6) → RUNNING (7) → UPLOADING (8)
→ COMPLETED (10), FAILED (11), or CANCELLED (12)
```

## Basic Streaming

```typescript
import { inference } from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

const stream = await client.run({
  app: 'google/veo-3-1-fast',
  input: { prompt: 'A sunset timelapse' }
}, { stream: true });

for await (const update of stream) {
  console.log(`Status: ${update.status}`);
}
```

## Handling Different Update Types

```typescript
const stream = await client.run(config, { stream: true });

for await (const update of stream) {
  const { status } = update;

  // Task state changes
  if (status === 'queued') {
    console.log('Task queued, waiting for worker...');
  } else if (status === 'running') {
    console.log('Task is running...');
  } else if (status === 'completed') {
    console.log('Done!');
    console.log('Output:', update.output);
  } else if (status === 'failed') {
    console.log('Error:', update.error);
  }

  // Progress logs
  if (update.logs?.length) {
    for (const log of update.logs) {
      console.log(`  Log: ${log}`);
    }
  }

  // Partial outputs
  if (update.partial_output) {
    console.log(`  Partial: ${update.partial_output}`);
  }
}
```

## Progress Tracking with UI

### Node.js CLI Progress Bar

```typescript
import { inference } from '@inferencesh/sdk';

function progressBar(current: number, total: number, width = 50) {
  const filled = Math.round(width * current / total);
  const bar = '█'.repeat(filled) + '░'.repeat(width - filled);
  const percent = (current / total * 100).toFixed(1);
  process.stdout.write(`\r[${bar}] ${percent}%`);
}

const stream = await client.run(config, { stream: true });

for await (const update of stream) {
  if (update.progress) {
    progressBar(update.progress.current, update.progress.total);
  }

  if (update.status === 'completed') {
    console.log('\n✓ Complete!');
  }
}
```

### React Progress Component

```typescript
import { useState, useEffect } from 'react';
import { inference } from '@inferencesh/sdk';

function ProgressDisplay({ config }: { config: any }) {
  const [status, setStatus] = useState('idle');
  const [progress, setProgress] = useState(0);
  const [logs, setLogs] = useState<string[]>([]);

  useEffect(() => {
    const client = inference({ proxyUrl: '/api/inference/proxy' });

    async function run() {
      const stream = await client.run(config, { stream: true });

      for await (const update of stream) {
        setStatus(update.status);

        if (update.progress) {
          setProgress(update.progress.current / update.progress.total * 100);
        }

        if (update.logs) {
          setLogs(prev => [...prev, ...update.logs]);
        }
      }
    }

    run();
  }, [config]);

  return (
    <div>
      <div>Status: {status}</div>
      <progress value={progress} max={100} />
      <ul>
        {logs.map((log, i) => <li key={i}>{log}</li>)}
      </ul>
    </div>
  );
}
```

## Streaming with Timeout

```typescript
async function streamWithTimeout(config: any, timeoutMs: number) {
  const client = inference({ [REDACTED_SECRET]' });
  const start = Date.now();

  const stream = await client.run(config, { stream: true });

  for await (const update of stream) {
    if (Date.now() - start > timeoutMs) {
      console.log('Timeout reached');
      break;
    }

    console.log(`Status: ${update.status}`);

    if (['completed', 'failed'].includes(update.status)) {
      return update;
    }
  }
}

const result = await streamWithTimeout(config, 60000); // 1 minute
```

## Agent Streaming

```typescript
const agent = client.agent('my-org/assistant@latest');

const response = await agent.sendMessage('Explain quantum entanglement', {
  onMessage: (msg) => {
    if (msg.content) {
      // Stream text as it arrives
      process.stdout.write(msg.content);
    }

    if (msg.type === 'thinking') {
      console.log(`\n[Thinking: ${msg.content}]`);
    }
  },
  onToolCall: async (call) => {
    console.log(`\n[Calling tool: ${call.name}]`);
    const result = await executeTool(call.name, call.args);
    agent.submitToolResult(call.id, result);
  }
});
```

## Multiple Streams in Parallel

```typescript
async function parallelStreams() {
  const client = inference({ [REDACTED_SECRET]' });

  const configs = [
    { app: 'infsh/flux-schnell', input: { prompt: 'A mountain' } },
    { app: 'infsh/flux-schnell', input: { prompt: 'An ocean' } },
    { app: 'infsh/flux-schnell', input: { prompt: 'A forest' } }
  ];

  async function streamOne(config: any, index: number) {
    const stream = await client.run(config, { stream: true });

    for await (const update of stream) {
      console.log(`[${index}] ${update.status}`);

      if (update.status === 'completed') {
        return update.output;
      }
    }
  }

  const results = await Promise.all(
    configs.map((c, i) => streamOne(c, i))
  );

  return results;
}
```

## Cancelling a Stream

```typescript
async function cancellableStream(config: any) {
  const client = inference({ [REDACTED_SECRET]' });
  const controller = new AbortController();

  // Cancel after 10 seconds
  setTimeout(() => controller.abort(), 10000);

  try {
    const stream = await client.run(config, {
      stream: true,
      signal: controller.signal
    });

    for await (const update of stream) {
      console.log(update.status);
    }
  } catch (e) {
    if (e.name === 'AbortError') {
      console.log('Stream cancelled');
    } else {
      throw e;
    }
  }
}
```

## Collecting All Logs

```typescript
async function collectLogs(config: any) {
  const client = inference({ [REDACTED_SECRET]' });
  const allLogs: string[] = [];

  const stream = await client.run(config, { stream: true });

  for await (const update of stream) {
    if (update.logs) {
      allLogs.push(...update.logs);
    }

    if (update.status === 'completed') {
      console.log('Final logs:');
      allLogs.forEach(log => console.log(`  ${log}`));
      return update.output;
    }
  }
}
```

## Custom Stream Processor Class

```typescript
class StreamProcessor {
  logs: string[] = [];
  startTime?: number;
  endTime?: number;

  process(update: any): boolean {
    if (!this.startTime) {
      this.startTime = Date.now();
    }

    if (update.logs) {
      this.logs.push(...update.logs);
    }

    if (['completed', 'failed'].includes(update.status)) {
      this.endTime = Date.now();
      return true; // Done
    }

    return false; // Continue
  }

  get duration(): number | null {
    if (this.startTime && this.endTime) {
      return this.endTime - this.startTime;
    }
    return null;
  }
}

// Usage
const processor = new StreamProcessor();
const stream = await client.run(config, { stream: true });

for await (const update of stream) {
  if (processor.process(update)) {
    break;
  }
}

console.log(`Duration: ${processor.duration}ms`);
console.log(`Logs: ${processor.logs.length}`);
```

## Server-Sent Events in Browser

```typescript
// For custom SSE handling in browser
async function browserSSE(taskId: string) {
  const eventSource = new EventSource(
    `/api/inference/stream?taskId=${taskId}`
  );

  eventSource.onmessage = (event) => {
    const update = JSON.parse(event.data);
    console.log('Update:', update);

    if (['completed', 'failed'].includes(update.status)) {
      eventSource.close();
    }
  };

  eventSource.onerror = (error) => {
    console.error('SSE error:', error);
    eventSource.close();
  };
}
```

## React Hook for Streaming

```typescript
import { useState, useCallback, useRef } from 'react';
import { inference } from '@inferencesh/sdk';

interface StreamState {
  status: string;
  output: any;
  logs: string[];
  error: string | null;
}

function useStream() {
  const [state, setState] = useState<StreamState>({
    status: 'idle',
    output: null,
    logs: [],
    error: null
  });
  const controllerRef = useRef<AbortController | null>(null);

  const run = useCallback(async (config: any) => {
    const client = inference({ proxyUrl: '/api/inference/proxy' });
    controllerRef.current = new AbortController();

    setState({ status: 'starting', output: null, logs: [], error: null });

    try {
      const stream = await client.run(config, {
        stream: true,
        signal: controllerRef.current.signal
      });

      for await (const update of stream) {
        setState(prev => ({
          ...prev,
          status: update.status,
          logs: update.logs ? [...prev.logs, ...update.logs] : prev.logs,
          output: update.output || prev.output
        }));
      }
    } catch (e: any) {
      if (e.name !== 'AbortError') {
        setState(prev => ({ ...prev, status: 'error', error: e.message }));
      }
    }
  }, []);

  const cancel = useCallback(() => {
    controllerRef.current?.abort();
  }, []);

  return { ...state, run, cancel };
}

// Usage in component
function Generator() {
  const { status, output, logs, run, cancel } = useStream();

  return (
    <div>
      <button onClick={() => run({ app: 'my-app', input: {...} })}>
        Start
      </button>
      <button onClick={cancel}>Cancel</button>
      <div>Status: {status}</div>
      {output && <img src={output.url} />}
    </div>
  );
}
```


path: references/tool-builder.md
content: # Tool Builder Reference

Complete guide to building tools with the JavaScript SDK.

## Parameter Types

### Basic Types

```typescript
import { string, number, integer, boolean } from '@inferencesh/sdk';

// String parameter
const name = string('The user\'s full name');

// Number (float)
const score = number('Score between 0 and 1');

// Integer
const count = integer('Number of items');

// Boolean
const enabled = boolean('Whether feature is enabled');
```

### Enum Type

```typescript
import { enumOf } from '@inferencesh/sdk';

const priority = enumOf(
  ['low', 'medium', 'high', 'critical'],
  'Task priority level'
);
```

### Array Type

```typescript
import { array, string, obj, integer } from '@inferencesh/sdk';

// Array of strings
const tags = array(string('Tag name'), 'List of tags');

// Array of objects
const items = array(
  obj({
    name: string('Item name'),
    qty: integer('Quantity')
  }),
  'List of items'
);
```

### Object Type

```typescript
import { obj, string, integer, optional } from '@inferencesh/sdk';

const address = obj({
  street: string('Street address'),
  city: string('City name'),
  state: string('State code'),
  zip: optional(string('ZIP code'))
}, 'Mailing address');
```

### Optional Parameters

```typescript
import { optional, string } from '@inferencesh/sdk';

// Optional string
const nickname = optional(string('User\'s nickname'));
```

## Tool Types

### Client Tools

Tools that execute in your code:

```typescript
import { tool, string, integer } from '@inferencesh/sdk';

// Basic tool
const greet = tool('greet')
  .describe('Greets a user')
  .param('name', string('Name to greet'))
  .build();

// Tool with multiple parameters
const sendEmail = tool('send_email')
  .display('Send Email')
  .describe('Sends an email to a recipient')
  .param('to', string('Recipient email'))
  .param('subject', string('Email subject'))
  .param('body', string('Email body'))
  .param('priority', integer('Priority 1-5'), 3) // default value
  .requireApproval()
  .build();
```

### App Tools

Tools that call inference.sh apps:

```typescript
import { appTool, string } from '@inferencesh/sdk';

// Basic app tool
const generate = appTool('generate_image', 'infsh/flux-schnell@latest')
  .describe('Generate an image from a text prompt')
  .param('prompt', string('Image description'))
  .build();

// App tool with setup and defaults
const translate = appTool('translate', 'infsh/translator@latest')
  .describe('Translate text between languages')
  .param('text', string('Text to translate'))
  .param('targetLang', string('Target language code'))
  .setup({
    model: 'advanced',
    preserveFormatting: true
  })
  .input({
    sourceLang: 'auto'
  })
  .build();
```

### Agent Tools

Tools that delegate to other agents:

```typescript
import { agentTool, string } from '@inferencesh/sdk';

const researcher = agentTool('research', 'my-org/researcher@v1')
  .describe('Research a topic in depth')
  .param('topic', string('Topic to research'))
  .param('depth', string('Research depth: brief, moderate, comprehensive'))
  .build();

const coder = agentTool('write_code', 'my-org/coder@latest')
  .describe('Write code to solve a problem')
  .param('task', string('Coding task description'))
  .param('language', string('Programming language'))
  .build();
```

### Webhook Tools

Tools that call external HTTP endpoints:

```typescript
import { webhookTool, string } from '@inferencesh/sdk';

// Slack notification
const slack = webhookTool('notify_slack', 'https://hooks.slack.com/services/...')
  .describe('Send a message to Slack')
  .param('channel', string('Channel name'))
  .param('message', string('Message text'))
  .build();

// Webhook with secret
const github = webhookTool('create_issue', 'https://api.github.com/repos/org/repo/issues')
  .describe('Create a GitHub issue')
  .secret('GITHUB_TOKEN') // Uses stored secret
  .param('title', string('Issue title'))
  .param('body', string('Issue description'))
  .build();
```

## Tool Builder Methods

### Common Methods

| Method | Description |
|--------|-------------|
| `.describe(text)` | Set tool description |
| `.display(name)` | Set display name |
| `.param(name, type, default?)` | Add parameter |
| `.requireApproval()` | Require human approval |
| `.build()` | Build the tool |

### App Tool Methods

| Method | Description |
|--------|-------------|
| `.setup(config)` | Hidden setup configuration |
| `.input(defaults)` | Default input values |

### Webhook Tool Methods

| Method | Description |
|--------|-------------|
| `.secret(name)` | Use stored secret for auth |

## Internal Tools

Built-in capabilities you can enable:

```typescript
import { internalTools } from '@inferencesh/sdk';

const config = internalTools()
  .plan()                    // Task planning
  .memory()                  // Information storage
  .webSearch(true)          // Web search capability
  .codeExecution(true)      // Run code
  .imageGeneration({
    enabled: true,
    appRef: 'infsh/flux@latest'
  })
  .build();
```

### Internal Tool Options

| Tool | Description |
|------|-------------|
| `.plan()` | Enable task breakdown and planning |
| `.memory()` | Enable information storage |
| `.webSearch(enabled)` | Enable/disable web search |
| `.codeExecution(enabled)` | Enable/disable code running |
| `.imageGeneration(config)` | Configure image generation |

## Handling Tool Calls

### Basic Handler

```typescript
const response = await agent.sendMessage('Greet John', {
  onToolCall: async (call) => {
    let result;

    if (call.name === 'greet') {
      result = `Hello, ${call.args.name}!`;
    } else if (call.name === 'calculate') {
      result = eval(call.args.expression);
    } else {
      result = { error: `Unknown tool: ${call.name}` };
    }

    agent.submitToolResult(call.id, result);
  }
});
```

### With Approval

```typescript
import * as readline from 'readline';

async function promptUser(question: string): Promise<boolean> {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
  });
  return new Promise(resolve => {
    rl.question(question, answer => {
      rl.close();
      resolve(answer.toLowerCase() === 'y');
    });
  });
}

const response = await agent.sendMessage('Delete temp files', {
  onToolCall: async (call) => {
    if (call.requiresApproval) {
      console.log(`Tool: ${call.name}`);
      console.log(`Args: ${JSON.stringify(call.args)}`);
      const approved = await promptUser('Approve? (y/n): ');

      if (!approved) {
        agent.submitToolResult(call.id, { error: 'Denied by user' });
        return;
      }
    }

    const result = await executeTool(call.name, call.args);
    agent.submitToolResult(call.id, result);
  }
});
```

### Widget Results

Return structured data for UI widgets:

```typescript
const response = await agent.sendMessage('Show order confirmation', {
  onToolCall: async (call) => {
    if (call.name === 'confirm_order') {
      // Return widget data
      agent.submitToolResult(call.id, {
        action: { type: 'confirm' },
        formData: {
          orderId: '12345',
          items: ['Widget A', 'Widget B'],
          total: 99.99
        }
      });
    }
  }
});
```

## TypeScript Types

```typescript
import type {
  AgentTool,
  ToolCall,
  ToolResult,
  ParamType
} from '@inferencesh/sdk';

// Type-safe tool definition
const typedTool: AgentTool = tool('my_tool')
  .describe('A typed tool')
  .param('input', string('Input value'))
  .build();

// Type-safe handler
function handleToolCall(call: ToolCall): ToolResult {
  return { success: true, data: call.args };
}
```

## Complete Example

```typescript
import {
  inference, tool, appTool, webhookTool,
  string, number, integer, boolean, enumOf,
  array, obj, optional, internalTools
} from '@inferencesh/sdk';

const client = inference({ [REDACTED_SECRET]' });

// Calculator tool
const calculator = tool('calculate')
  .display('Calculator')
  .describe('Perform mathematical calculations')
  .param('expression', string('Math expression to evaluate'))
  .build();

// Image generation tool
const imageGen = appTool('generate_image', 'infsh/flux-schnell@latest')
  .describe('Generate an image from text')
  .param('prompt', string('Image description'))
  .param('style', enumOf(['realistic', 'artistic', 'cartoon'], 'Image style'))
  .setup({ quality: 'high' })
  .input({ steps: 20 })
  .requireApproval()
  .build();

// Slack notification tool
const slack = webhookTool('notify', 'https://hooks.slack.com/...')
  .describe('Send Slack notification')
  .param('message', string('Message to send'))
  .build();

// Built-in tools
const internals = internalTools()
  .webSearch(true)
  .codeExecution(true)
  .build();

// Create agent with all tools
const agent = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: 'You are a helpful assistant with various capabilities.',
  tools: [calculator, imageGen, slack],
  internal_tools: internals,
  temperature: 0.7
});

// Handle tool calls
const response = await agent.sendMessage(
  'Calculate 15% tip on $85, then notify Slack',
  {
    onToolCall: async (call) => {
      if (call.name === 'calculate') {
        try {
          const result = eval(call.args.expression);
          agent.submitToolResult(call.id, { result });
        } catch (e) {
          agent.submitToolResult(call.id, { error: String(e) });
        }
      } else if (call.requiresApproval) {
        const approved = await promptUser(`Allow ${call.name}? (y/n): `);
        if (!approved) {
          agent.submitToolResult(call.id, { error: 'Denied' });
        }
        // App/webhook tools execute automatically if approved
      }
    }
  }
);
```

## Server Proxy Tools

When using the SDK in the browser, tools that require API keys should use webhooks with secrets stored server-side:

```typescript
// Browser-safe tool definition
const apiTool = webhookTool('call_api', '/api/my-endpoint')
  .describe('Call server-side API')
  .param('data', obj({
    action: string('Action to perform'),
    payload: optional(string('Optional payload'))
  }))
  .build();

// Server-side handler (e.g., Next.js API route)
// app/api/my-endpoint/route.ts
export async function POST(req: Request) {
  const { data } = await req.json();

  // Access secrets safely on server
  const result = await fetch(process.env.EXTERNAL_API_URL, {
    headers: { Authorization: `Bearer ${process.env.API_SECRET}` },
    body: JSON.stringify(data)
  });

  return Response.json(await result.json());
}
```


path: references/typescript.md
content: # TypeScript Reference

Full type definitions and type-safe patterns.

## Installation

TypeScript definitions are included with the SDK:

```bash
npm install @inferencesh/sdk
```

No additional `@types` package needed.

## Basic Types

```typescript
import {
  inference,
  type TaskDTO,
  type ChatDTO,
  type ChatMessageDTO,
  type AgentTool,
  type TaskStatus
} from '@inferencesh/sdk';
```

## Task Types

```typescript
import type {
  TaskDTO,
  TaskStatus,
  TaskStatusCompleted,
  TaskStatusFailed,
  TaskStatusRunning,
  TaskOutput
} from '@inferencesh/sdk';

// Check task status
function handleTask(task: TaskDTO) {
  if (task.status === 'completed') {
    console.log('Output:', task.output);
  } else if (task.status === 'failed') {
    console.log('Error:', task.error);
  } else if (task.status === 'running') {
    console.log('Progress:', task.progress);
  }
}
```

## Run Options Types

```typescript
import type { RunOptions, StreamOptions } from '@inferencesh/sdk';

const options: RunOptions = {
  wait: true,
  stream: false
};

const streamOptions: StreamOptions = {
  stream: true,
  signal: new AbortController().signal
};
```

## Agent Types

```typescript
import type {
  Agent,
  AgentConfig,
  AgentMessage,
  AgentToolCall,
  AgentResponse
} from '@inferencesh/sdk';

// Type-safe agent config
const config: AgentConfig = {
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: 'You are a helpful assistant.',
  tools: [],
  temperature: 0.7,
  max_tokens: 4096
};

// Type-safe message handler
function handleMessage(msg: AgentMessage) {
  if (msg.type === 'text') {
    console.log(msg.content);
  } else if (msg.type === 'tool_use') {
    console.log('Tool:', msg.tool_name);
  }
}
```

## Tool Types

```typescript
import {
  tool, string, number, integer, boolean, enumOf, array, obj, optional,
  type AgentTool,
  type ParamType,
  type ToolCall
} from '@inferencesh/sdk';

// Type-safe tool definition
const myTool: AgentTool = tool('my_tool')
  .describe('Does something')
  .param('input', string('Input value'))
  .build();

// Type-safe tool call handler
function handleToolCall(call: ToolCall) {
  const { id, name, args } = call;
  console.log(`Tool ${name} called with:`, args);
}
```

## Parameter Type Helpers

```typescript
import type { ParamType, StringParam, NumberParam, ObjectParam } from '@inferencesh/sdk';

// String parameter
const nameParam: ParamType = string('User name');

// Number parameter
const scoreParam: ParamType = number('Score value');

// Object parameter
const addressParam: ObjectParam = obj({
  street: string('Street'),
  city: string('City'),
  zip: optional(string('ZIP'))
}, 'Address');
```

## Generic Client

```typescript
import { inference, type InferenceClient } from '@inferencesh/sdk';

const client: InferenceClient = inference({ [REDACTED_SECRET]' });

// All methods are typed
const result = await client.run({
  app: 'my-app',
  input: { data: 'test' }
});
// result is TaskDTO
```

## Custom Input/Output Types

```typescript
interface MyAppInput {
  prompt: string;
  style?: 'realistic' | 'artistic' | 'cartoon';
  steps?: number;
}

interface MyAppOutput {
  image: string;
  metadata: {
    seed: number;
    duration_ms: number;
  };
}

async function generateImage(input: MyAppInput): Promise<MyAppOutput> {
  const result = await client.run({
    app: 'infsh/flux-schnell',
    input
  });

  return result.output as MyAppOutput;
}
```

## Type Guards

```typescript
import type { TaskDTO, TaskStatus } from '@inferencesh/sdk';

function isCompleted(task: TaskDTO): task is TaskDTO & { status: 'completed' } {
  return task.status === 'completed';
}

function isFailed(task: TaskDTO): task is TaskDTO & { status: 'failed' } {
  return task.status === 'failed';
}

// Usage
const task = await client.run({ app: 'my-app', input: {} });

if (isCompleted(task)) {
  // TypeScript knows task.output exists here
  console.log(task.output);
} else if (isFailed(task)) {
  // TypeScript knows task.error exists here
  console.log(task.error);
}
```

## Error Types

```typescript
import {
  InferenceError,
  RequirementsNotMetException,
  type RequirementError
} from '@inferencesh/sdk';

try {
  await client.run({ app: 'my-app', input: {} });
} catch (e) {
  if (e instanceof RequirementsNotMetException) {
    const errors: RequirementError[] = e.errors;
    for (const err of errors) {
      console.log(`${err.type}: ${err.key}`);
    }
  } else if (e instanceof InferenceError) {
    console.log('API error:', e.message);
  }
}
```

## File Types

```typescript
import type { FileDTO, UploadOptions } from '@inferencesh/sdk';

const options: UploadOptions = {
  filename: 'image.png',
  contentType: 'image/png',
  public: true
};

const file: FileDTO = await client.uploadFile('/path/to/file', options);
console.log(file.uri, file.url, file.size);
```

## Stream Types

```typescript
import type { StreamUpdate, StreamOptions } from '@inferencesh/sdk';

async function* typedStream(config: any): AsyncGenerator<StreamUpdate> {
  const stream = await client.run(config, { stream: true });
  yield* stream;
}

// Usage
for await (const update of typedStream({ app: 'my-app', input: {} })) {
  // update is StreamUpdate
  console.log(update.status);
}
```

## React Integration Types

```typescript
import type { Agent, AgentMessage, ToolCall } from '@inferencesh/sdk';

interface ChatState {
  messages: AgentMessage[];
  loading: boolean;
  error: string | null;
}

interface ChatActions {
  sendMessage: (text: string) => Promise<void>;
  reset: () => void;
}

function useChat(agentRef: string): ChatState & ChatActions {
  // Implementation
}
```

## Strict Mode Configuration

For strictest type checking, use these tsconfig options:

```json
{
  "compilerOptions": {
    "strict": true,
    "noUncheckedIndexedAccess": true,
    "exactOptionalPropertyTypes": true
  }
}
```

## Module Augmentation

Extend types for custom use cases:

```typescript
import '@inferencesh/sdk';

declare module '@inferencesh/sdk' {
  interface TaskDTO {
    customField?: string;
  }
}
```

## JSDoc Type Annotations

For JavaScript projects wanting type hints:

```javascript
/** @type {import('@inferencesh/sdk').InferenceClient} */
const client = inference({ [REDACTED_SECRET]' });

/**
 * @param {import('@inferencesh/sdk').TaskDTO} task
 */
function handleTask(task) {
  console.log(task.status);
}
```

## Zod Integration

Validate runtime types with Zod:

```typescript
import { z } from 'zod';

const ImageOutputSchema = z.object({
  image: z.string().url(),
  metadata: z.object({
    seed: z.number(),
    duration_ms: z.number()
  })
});

type ImageOutput = z.infer<typeof ImageOutputSchema>;

async function generateImage(prompt: string): Promise<ImageOutput> {
  const result = await client.run({
    app: 'infsh/flux-schnell',
    input: { prompt }
  });

  // Validate at runtime
  return ImageOutputSchema.parse(result.output);
}
```

## Full Example

```typescript
import {
  inference,
  tool,
  appTool,
  string,
  enumOf,
  type InferenceClient,
  type AgentTool,
  type TaskDTO,
  type ToolCall
} from '@inferencesh/sdk';

// Typed client
const client: InferenceClient = inference({
  [REDACTED_SECRET]
});

// Typed tools
const tools: AgentTool[] = [
  tool('search')
    .describe('Search for information')
    .param('query', string('Search query'))
    .build(),

  appTool('generate', 'infsh/flux-schnell@latest')
    .describe('Generate image')
    .param('prompt', string('Description'))
    .param('style', enumOf(['realistic', 'artistic'], 'Style'))
    .build()
];

// Typed agent
const agent = client.agent({
  core_app: { ref: 'infsh/claude-sonnet-4@latest' },
  system_prompt: 'You are helpful.',
  tools
});

// Typed handler
async function handleToolCall(call: ToolCall): Promise<void> {
  console.log(`Tool: ${call.name}, Args:`, call.args);
}

// Typed response handling
const response = await agent.sendMessage('Hello', {
  onToolCall: handleToolCall
});

console.log(response.text);
```


Security scan verdicts — ClawScan: suspicious | static analysis: suspicious | VirusTotal: suspicious | SkillSpector: suspicious (CRITICAL)