npm package: @kognitivedev/vercel-ai-provider
Description: Vercel AI SDK middleware for memory injection and prompt management
Latest version: 0.2.22

# @kognitivedev/vercel-ai-provider

Vercel AI SDK provider wrapper that integrates the Kognitive memory layer into your AI applications. Automatically injects memory context and logs conversations for memory processing.

## Installation

```bash
npm install @kognitivedev/vercel-ai-provider
```

### Peer Dependencies

This package requires the Vercel AI SDK:

```bash
npm install ai
```

## Quick Start

```typescript
import { createCognitiveLayer } from "@kognitivedev/vercel-ai-provider";
import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";

// 1. Create the cognitive layer
const cl = createCognitiveLayer({
    provider: openai,
    clConfig: {
        appId: "my-app",
        defaultAgentId: "assistant",
        baseUrl: "http://localhost:3001"
    }
});

// 2. Use it with Vercel AI SDK
const { text } = await generateText({
    model: cl("gpt-4o", {
        userId: "user-123",
        sessionId: "session-abc"
    }),
    prompt: "What's my favorite color?"
});
```

## Configuration

### `CognitiveLayerConfig`

| Option | Type | Required | Default | Description |
|--------|------|----------|---------|-------------|
| `appId` | `string` | Yes | - | Unique identifier for your application |
| `defaultAgentId` | `string` | - | `"default"` | Default agent ID when not specified per-request |
| `baseUrl` | `string` | - | `"http://localhost:3001"` | Kognitive backend API URL |
| `apiKey` | `string` | - | - | API key for authentication (if required) |
| `processDelayMs` | `number` | - | `500` | Delay before triggering memory processing (set to 0 to disable) |
| `logLevel` | `LogLevel` | - | `'info'` | Controls console logging verbosity |
| `providerFactory` | `(baseURL: string, apiKey: string) => (modelId: string) => LanguageModel` | - | - | Factory for creating a provider that routes through a gateway URL |

### `LogLevel`

Controls verbosity of console output from the provider.

| Level | Description |
|-------|-------------|
| `'none'` | No logging |
| `'error'` | Only errors |
| `'warn'` | Errors and warnings |
| `'info'` | Errors, warnings, and info messages (default) |
| `'debug'` | All messages including detailed snapshot data |

```typescript
const cl = createCognitiveLayer({
    provider: openai,
    clConfig: {
        appId: "my-app",
        logLevel: "debug" // see full snapshot data in console
    }
});
```

## API Reference

### `createCognitiveLayer(config)`

Creates a cognitive layer that adds memory capabilities to any Vercel AI SDK provider.

**Parameters:**

```typescript
createCognitiveLayer({
    provider: any,        // Vercel AI SDK provider (e.g., openai, anthropic)
    clConfig: CognitiveLayerConfig
}): CognitiveLayer
```

**Returns:** `CognitiveLayer` - A callable model wrapper with `.streamText()` and `.generateText()` methods attached.

---

### `CognitiveLayer`

The return type of `createCognitiveLayer`. It is callable as a model wrapper and also exposes prompt-driven helpers.

```typescript
type CognitiveLayer = CLModelWrapper & {
    streamText: (options: CLStreamTextOptions) => Promise<ReturnType<typeof streamText>>;
    generateText: (options: CLGenerateTextOptions) => ReturnType<typeof generateText>;
    resolvePrompt: (slug: string) => Promise<CachedPrompt>;
    logConversation: (payload: LogConversationPayload) => Promise<void>;
    triggerProcessing: (userId: string, projectId: string, sessionId: string) => void;
    clearSessionCache: (sessionKey?: string) => void;
};
```

---

### `CLModelWrapper`

The callable signature of `CognitiveLayer`.

```typescript
type CLModelWrapper = (
    modelId: string,
    settings?: {
        userId?: string;
        agentId?: string;
        sessionId?: string;
    },
    providerOptions?: Record<string, unknown>
) => LanguageModel;
```

**Parameters:**

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `modelId` | `string` | Yes | Model identifier (e.g., `"gpt-4o"`, `"claude-3-opus"`) |
| `settings.userId` | `string` | - | User identifier (required for memory features) |
| `settings.agentId` | `string` | - | Override default agent ID |
| `settings.sessionId` | `string` | - | Session identifier (required for logging) |
| `providerOptions` | `Record<string, unknown>` | - | Provider-specific options passed directly to the underlying provider |

---

### `PromptConfig`

Used with `cl.streamText()` and `cl.generateText()` to reference a managed prompt.

```typescript
interface PromptConfig {
    slug: string;
    variables?: Record<string, string | boolean>;
}
```

## Usage Examples

### With OpenAI

```typescript
import { createCognitiveLayer } from "@kognitivedev/vercel-ai-provider";
import { openai } from "@ai-sdk/openai";
import { generateText } from "ai";

const cl = createCognitiveLayer({
    provider: openai,
    clConfig: {
        appId: "my-app",
        baseUrl: "https://api.kognitive.dev"
    }
});

const { text } = await generateText({
    model: cl("gpt-4o", {
        userId: "user-123",
        sessionId: "session-abc"
    }),
    prompt: "Remember that my favorite color is blue"
});
```

### With OpenRouter (Provider Options)

Pass provider-specific options as the third parameter:

```typescript
import { createCognitiveLayer } from "@kognitivedev/vercel-ai-provider";
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { generateText } from "ai";

const openrouter = createOpenRouter({
    apiKey: process.env.OPENROUTER_API_KEY
});

const cl = createCognitiveLayer({
    provider: openrouter.chat,
    clConfig: {
        appId: "my-app",
        baseUrl: "https://api.kognitive.dev"
    }
});

// Pass provider-specific options as the third parameter
const { text } = await generateText({
    model: cl("moonshotai/kimi-k2-0905", {
        userId: "user-123",
        sessionId: "session-abc"
    }, {
        provider: {
            only: ["openai"]
        }
    }),
    prompt: "What's the weather like?"
});
```

### With Anthropic

```typescript
import { createCognitiveLayer } from "@kognitivedev/vercel-ai-provider";
import { anthropic } from "@ai-sdk/anthropic";
import { streamText } from "ai";

const cl = createCognitiveLayer({
    provider: anthropic,
    clConfig: {
        appId: "my-app",
        defaultAgentId: "claude-assistant"
    }
});

const result = await streamText({
    model: cl("claude-3-5-sonnet-latest", {
        userId: "user-456",
        sessionId: "chat-xyz"
    }),
    prompt: "What did I tell you about my favorite color?"
});

for await (const chunk of result.textStream) {
    process.stdout.write(chunk);
}
```

### With System Prompts

The provider automatically injects memory context into your system prompts:

```typescript
const { text } = await generateText({
    model: cl("gpt-4o", {
        userId: "user-123",
        sessionId: "session-abc"
    }),
    system: "You are a helpful assistant.",
    prompt: "What do you know about me?"
});

// Memory context is automatically appended to system prompt
```

### Prompt Management

Use `cl.streamText()` and `cl.generateText()` to resolve prompts by slug from the backend. Prompts are fetched from the `/api/cognitive/prompt` endpoint and cached for 60 seconds.

Prompt templates use [Handlebars](https://handlebarsjs.com/) syntax, giving you variable interpolation and conditional blocks. This means prompts can be fully managed from the dashboard without any string concatenation in application code.

#### Template Syntax

**Variable interpolation** — insert a value by name:

```handlebars
Hello {{userName}}, welcome to {{companyName}}.
```

**Conditional blocks** — include content only when a variable is truthy:

```handlebars
{{#if hasImages}}
The user has attached images. Please analyze them.
{{/if}}
```

**If / else** — provide a fallback when the condition is falsy:

```handlebars
{{#if isPremium}}
You have access to all features.
{{else}}
Upgrade to premium for full access.
{{/if}}
```

**Unless** — inverse of `if`, includes the block when the variable is falsy:

```handlebars
{{#unless hasHistory}}
This is a new user. Introduce yourself.
{{/unless}}
```

**Nesting** — conditionals can be nested arbitrarily:

```handlebars
{{#if hasAttachments}}
  Attachments provided.
  {{#if hasImages}}
    Includes images for visual analysis.
  {{/if}}
{{/if}}
```

#### Variable Types

Variables can be `string` or `boolean`:

| Type | Truthy | Falsy |
|------|--------|-------|
| `string` | Any non-empty string | `""` (empty string) |
| `boolean` | `true` | `false` |

Undefined variables render as empty string in interpolation and are treated as falsy in conditionals.

> **Note:** Content is not HTML-escaped — prompt text passes through as-is, which is the correct behavior for LLM prompts.

#### Usage with `cl.streamText()` / `cl.generateText()`

Pass variables in the `prompt.variables` field:

```typescript
// Stream with a managed prompt
const result = await cl.streamText({
    model: cl("gpt-4o", {
        userId: "user-123",
        sessionId: "session-abc"
    }),
    prompt: {
        slug: "fitness-coach",
        variables: {
            userName: "Sarah",
            fitnessGoal: "Build muscle",
            hasImages: true,
            hasAttachments: false
        }
    },
    messages: [{ role: "user", content: "Check my squat form" }]
});

for await (const chunk of result.textStream) {
    process.stdout.write(chunk);
}
```

```typescript
// Generate with a managed prompt (string-only variables work too)
const { text } = await cl.generateText({
    model: cl("gpt-4o", {
        userId: "user-123",
        sessionId: "session-abc"
    }),
    prompt: {
        slug: "summarizer",
        variables: {
            language: "English"
        }
    },
    messages: [{ role: "user", content: "Summarize this document..." }]
});
```

#### Standalone `renderTemplate()`

The template engine is also exported directly for use outside of prompt management:

```typescript
import { renderTemplate } from "@kognitivedev/vercel-ai-provider";

const result = renderTemplate(
    "Hello {{name}}! {{#if isVip}}Welcome back, VIP.{{/if}}",
    { name: "Alice", isVip: true }
);
// → "Hello Alice! Welcome back, VIP."
```

#### Full Example Template

A prompt template stored in the dashboard might look like:

```handlebars
You are a fitness coach AI assistant.

User: {{userName}}
Goal: {{fitnessGoal}}

{{#if hasImages}}
The user has attached images for form analysis. Please review them carefully.
{{/if}}
{{#if hasAttachments}}
Additional documents have been provided for context.
{{/if}}
{{#unless hasHistory}}
This is a new user with no prior conversation history. Introduce yourself.
{{/unless}}

Please provide personalized advice.
```

### Gateway Routing

When a prompt has a `gatewaySlug` configured on the backend, requests are automatically routed through the gateway endpoint at `{baseUrl}/api/cognitive/gateway/{gatewaySlug}`.

To enable gateway routing, provide a `providerFactory` in your config:

```typescript
import { createOpenAI } from "@ai-sdk/openai";

const cl = createCognitiveLayer({
    provider: openai,
    clConfig: {
        appId: "my-app",
        baseUrl: "https://api.kognitive.dev",
        providerFactory: (baseURL, apiKey) => {
            const gatewayProvider = createOpenAI({ baseURL, apiKey });
            return (modelId) => gatewayProvider(modelId);
        }
    }
});
```

Without `providerFactory`, the provider will log a warning and skip gateway routing, sending requests directly to the original provider.

### `cl.resolvePrompt(slug)`

Fetches and caches a prompt by slug from the backend. Prompts are cached for 60 seconds.

```typescript
const prompt = await cl.resolvePrompt("customer-support");
console.log(prompt.content);   // The prompt template text
console.log(prompt.version);   // Prompt version number
console.log(prompt.promptId);  // Unique prompt ID
```

**Returns:** `Promise<CachedPrompt>`

```typescript
interface CachedPrompt {
    promptId: string;
    slug: string;
    version: number;
    content: string;
    fetchedAt: number;
    gatewaySlug?: string;
}
```

---

### `cl.logConversation(payload)`

Manually logs a conversation to the backend for memory processing.

```typescript
await cl.logConversation({
    userId: "user-123",
    projectId: "my-project",
    sessionId: "session-abc",
    messages: [
        { role: "user", content: "Hello" },
        { role: "assistant", content: "Hi there!" }
    ],
    modelId: "gpt-4o",
});
```

**Parameters:** `LogConversationPayload`

```typescript
interface LogConversationPayload {
    userId: string;
    projectId: string;
    sessionId: string;
    messages: any[];
    memorySystemPrompt?: string;
    modelId?: string;
    usage?: Record<string, unknown>;
    promptSlug?: string;
    promptVersion?: number;
    promptId?: string;
}
```

---

### `cl.triggerProcessing(userId, projectId, sessionId)`

Manually triggers memory processing for a given session. This is normally called automatically after logging, but can be invoked independently.

```typescript
cl.triggerProcessing("user-123", "my-project", "session-abc");
```

---

### `cl.clearSessionCache(sessionKey?)`

Clears cached memory snapshots. Pass a specific session key (`"userId:projectId:sessionId"`) to clear one session, or omit to clear all.

```typescript
// Clear a specific session
cl.clearSessionCache("user-123:my-project:session-abc");

// Clear all cached sessions
cl.clearSessionCache();
```

---

### Without Memory (Anonymous Users)

Skip memory features by omitting `userId`:

```typescript
const { text } = await generateText({
    model: cl("gpt-4o"),
    prompt: "General question without memory"
});
```

## How It Works

### Memory Injection Flow

1. **Request Interception**: When a request is made, the middleware fetches the user's memory snapshot
2. **Context Injection**: Memory context is injected into the system prompt as `<MemoryContext>` block
3. **Response Processing**: After the response, the conversation is logged
4. **Background Processing**: Memory extraction and management runs asynchronously

### Memory Context Format

The injected memory context follows this structure:

```xml
<MemoryContext>
Use the following memory to stay consistent. Prefer UserContext facts for answers; AgentHeuristics guide style, safety, and priorities.
<AgentHeuristics>
- User prefers concise responses
- Always greet user by name
</AgentHeuristics>
<UserContext>
<Facts>
- User's name is John
- Favorite color is blue
</Facts>
<State>
- Currently working on a project
</State>
</UserContext>
</MemoryContext>
```

## Backend API Integration

The provider communicates with your Kognitive backend via these endpoints:

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/cognitive/snapshot` | GET | Fetches user's memory snapshot |
| `/api/cognitive/log` | POST | Logs conversation for processing |
| `/api/cognitive/process` | POST | Triggers memory extraction/management |
| `/api/cognitive/prompt` | GET | Resolves a managed prompt by slug |
| `/api/cognitive/gateway/{slug}` | * | Gateway proxy endpoint for routed requests |

### Query Parameters for Snapshot

```
GET /api/cognitive/snapshot?userId={userId}&agentId={agentId}&appId={appId}
```

## Troubleshooting

### Memory not being injected

1. Ensure `userId` and `sessionId` are provided
2. Check that the backend is running at the configured `baseUrl`
3. Verify the snapshot endpoint returns data

### Console warnings

```
CognitiveLayer: sessionId is required to log and process memories; skipping logging until provided.
```

This warning appears when `userId` is provided but `sessionId` is missing. Add `sessionId` to enable logging.

### Processing delay

The default 500ms delay before triggering memory processing allows database writes to settle. Adjust with `processDelayMs`:

```typescript
clConfig: {
    processDelayMs: 1000 // 1 second delay
    // processDelayMs: 0 // Immediate processing
}
```

### Gateway not routing

If requests are not being routed through the gateway:

1. Ensure `providerFactory` is set in your `clConfig`
2. Verify the prompt has a `gatewaySlug` configured on the backend
3. Set `logLevel: 'debug'` to inspect gateway resolution logs

## License

MIT