npm package: @lens-os/sdk
Description: Lens OS Frontend AI Agent SDK
Author: Lens OS
Latest version: 0.1.5

# @lens-os/sdk

AI Agent SDK for building conversational AI experiences in web applications.

[![npm version](https://badge.fury.io/js/@lens-os/sdk.svg)](https://www.npmjs.com/package/@lens-os/sdk)

## Features

- **Server/Client Architecture** - API keys stay on the server, frontend communicates via SSE
- **Multi-turn Agent Loop** - Autonomous LLM orchestration with tool execution
- **Streaming Responses** - Real-time text and tool call events via Server-Sent Events
- **React Hooks** - `useLensAgent` and `useChat` for easy frontend integration
- **Tool System** - 3-tier priority: Manual executors > Customer endpoints > Platform built-ins
- **Session Management** - Conversation history with automatic memory compaction
- **Custom Tools** - Define your own tool executors with full metadata for prompt generation
- **Debug Mode** - Configurable logging via `debug` flag
- **TypeScript** - Full type definitions included

## Installation

```bash
npm install @lens-os/sdk
# or
bun add @lens-os/sdk
```

**Peer dependencies:**
- `openai` >= 4.0.0 (required)
- `react` >= 18.0.0 (optional, only for React hooks)

## Architecture

```
Browser (React)                          Server (Next.js / Node)
┌──────────────────┐                    ┌──────────────────────┐
│  useLensAgent()  │── POST /agent ───> │  createAgentHandler  │
│                  │<── SSE stream ──── │    SupervisorAgent   │
│                  │── POST /action ──> │    (LLM + Tools)     │
└──────────────────┘                    └──────────────────────┘
```

- **Frontend** sends messages and receives streamed responses
- **Server** holds API keys, runs the agent loop, calls LLM and tools
- **Action requests** allow the server to ask the client to perform DOM operations (click, scroll, navigate, etc.)

## Quick Start

### 1. Server-side: Create API Route

```typescript
// app/api/agent/route.ts (Next.js App Router)
import { createAgentHandler } from '@lens-os/sdk/server';

const handler = createAgentHandler({
  apiKey: process.env.LENS_API_KEY!,
  openaiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-4o',          // optional, default: gpt-4o
  debug: true,               // optional, enables SDK logging
});

export const POST = handler.POST;
```

### 2. Server-side: Create Action Result Route

```typescript
// app/api/agent/action/route.ts
import { createActionResultHandler } from '@lens-os/sdk/server';

// Share the pendingActions store from the agent handler
import { handler } from '../route';

const actionHandler = createActionResultHandler(handler._pendingActions);
export const POST = actionHandler.POST;
```

### 3. Client-side: React Hook

```tsx
import { useLensAgent } from '@lens-os/sdk/react';

function ChatWidget() {
  const {
    messages,
    isLoading,
    sendMessage,
    abort,
  } = useLensAgent({
    endpoint: '/api/agent',
  });

  return (
    <div>
      {messages.map((msg, i) => (
        <div key={i} className={msg.role}>
          {typeof msg.content === 'string' ? msg.content : '...'}
        </div>
      ))}
      <input
        onKeyDown={(e) => {
          if (e.key === 'Enter') {
            sendMessage(e.currentTarget.value);
            e.currentTarget.value = '';
          }
        }}
        placeholder="Type a message..."
        disabled={isLoading}
      />
      {isLoading && <button onClick={abort}>Stop</button>}
    </div>
  );
}
```

## Entry Points

| Import path | Usage |
|---|---|
| `@lens-os/sdk` | Core types, `LensClient`, `SupervisorAgent`, utilities |
| `@lens-os/sdk/react` | React hooks (`useLensAgent`, `useChat`) |
| `@lens-os/sdk/server` | Server handler (`createAgentHandler`, `createActionResultHandler`) |

## Server Configuration

```typescript
import { createAgentHandler } from '@lens-os/sdk/server';

const handler = createAgentHandler({
  // Required
  apiKey: string,              // Lens OS API key
  openaiKey: string,           // OpenAI API key

  // Optional
  baseUrl?: string,            // Default: https://osapi.ask-lens.ai
  model?: string,              // Default: gpt-4o
  maxTurns?: number,           // Default: 10
  language?: 'zh-TW' | 'en-US',
  debug?: boolean,             // Enable SDK logging (default: false)
  actionTimeout?: number,      // Action request timeout ms (default: 30000)

  // Shared client (reuse across handlers to share config cache)
  client?: LensClient,

  // Callbacks
  onTrace?: (trace: LLMTrace) => void,

  // Custom tool executors
  toolExecutors?: Record<string, ToolExecutorFunction | ToolExecutorConfig>,
});
```

## React Hooks

### `useLensAgent`

Full-featured hook for building chat interfaces.

```typescript
import { useLensAgent } from '@lens-os/sdk/react';

const {
  // State
  messages,           // Message[] - conversation history
  isLoading,          // boolean - agent is running
  sessionId,          // string - current session ID
  error,              // Error | null

  // Actions
  sendMessage,        // (message: string, context?) => Promise<void>
  abort,              // () => void - cancel current execution
  clearMessages,      // () => void - clear local messages
  newSession,         // () => void - start a new session
  loadSession,        // (sessionId: string, messages: Message[]) => void
} = useLensAgent({
  endpoint: '/api/agent',

  // Optional
  actionResultEndpoint?: string,    // Custom action result endpoint
  userId?: string,
  headers?: Record<string, string> | (() => Record<string, string>),
  onEvent?: (event: SSEEvent) => void,
  getPageState?: () => Promise<PageState>,
  onActionRequest?: (action: string, params: Record<string, any>) => Promise<ToolResult>,
});
```

### `useChat`

Simplified wrapper with shorter method names.

```typescript
import { useChat } from '@lens-os/sdk/react';

const {
  messages,
  isLoading,
  error,
  send,     // sendMessage
  stop,     // abort
  clear,    // clearMessages
  reset,    // newSession
} = useChat({
  endpoint: '/api/agent',
});
```

## Custom Tool Executors

Define custom tools on the server. Tool metadata is automatically injected into the LLM prompt.

```typescript
const handler = createAgentHandler({
  apiKey: '...',
  openaiKey: '...',
  toolExecutors: {
    // Simple function
    my_tool: async (params, context) => {
      return { success: true, result: 'done' };
    },

    // Full config with metadata (recommended)
    order_lookup: {
      description: 'Look up order details by order ID',
      whenToUse: 'User asks about order status or delivery',
      schema: {
        orderId: { type: 'string', required: true, description: 'The order ID' },
      },
      output: 'Order object with status, items, tracking info',
      execute: async (params, context) => {
        const order = await db.orders.findById(params.orderId);
        if (!order) {
          return { success: false, error: 'Order not found' };
        }
        return { success: true, result: order };
      },
    },
  },
});
```

### Tool Execution Priority

When the LLM calls a tool, the SDK resolves it in this order:

1. **Manual `toolExecutors`** - Code-defined executors (highest priority)
2. **CUSTOMER mode** - Database-configured external endpoint (POST to `customerEndpoint`)
3. **PLATFORM mode** - Built-in SDK implementations (`knowledge_search`, DOM tools, etc.)

## Shared LensClient

When running multiple handlers or agents, share a `LensClient` instance to reuse the config cache (60s TTL with automatic retry):

```typescript
import { LensClient } from '@lens-os/sdk';
import { createAgentHandler } from '@lens-os/sdk/server';

const sharedClient = new LensClient({
  apiKey: process.env.LENS_API_KEY!,
  baseUrl: 'https://osapi.ask-lens.ai',
});

const handler = createAgentHandler({
  apiKey: process.env.LENS_API_KEY!,
  openaiKey: process.env.OPENAI_API_KEY!,
  client: sharedClient,  // Reuses config cache
});
```

## Debug Mode

By default, the SDK produces no console output. Enable debug logging:

```typescript
// Via config
const handler = createAgentHandler({
  apiKey: '...', openaiKey: '...',
  debug: true,
});

// Or manually
import { setDebug } from '@lens-os/sdk';
setDebug(true);
```

## SSE Event Types

The server streams these events to the client:

| Event type | Description |
|---|---|
| `text` | Streamed text content from the LLM |
| `tool_call` | LLM is invoking a tool (name + parameters) |
| `tool_result` | Tool execution result |
| `action_request` | Server asks client to perform a DOM action |
| `error` | Error occurred (may be fatal) |
| `done` | Agent execution complete |

## Page State & DOM Actions

The SDK supports multimodal context. Send page state with messages:

```typescript
sendMessage('What products are on this page?', {
  pageState: {
    url: window.location.href,
    title: document.title,
    markdown: '...',           // Page content as markdown
    screenshot: 'data:...',    // Base64 screenshot
    actionableElements: [],    // Clickable elements
  },
});
```

The server can request DOM actions on the client (click, scroll, navigate, etc.) via the action request protocol. Handle them with `onActionRequest`:

```typescript
useLensAgent({
  endpoint: '/api/agent',
  onActionRequest: async (action, params) => {
    // Execute DOM action and return result
    return { success: true, result: 'clicked' };
  },
});
```

## Requirements

- Node.js >= 18
- React >= 18 (optional, for React hooks)
- OpenAI API key
- Lens OS API key

## License

MIT