npm package: @northslopetech/foundry-ai-sdk
Description: Custom AI SDK provider for Palantir Foundry Language Model Service
Author: Northslope Technologies
Latest version: 0.2.0

# Foundry AI SDK Provider

A custom AI SDK provider for Palantir Foundry Language Model Service, enabling seamless integration with Vercel's AI SDK.

## Features

✅ **Multiple Model Support**
- GPT models (GPT-4, GPT-5) with full tool calling support
- Gemini models (Gemini 2.0/2.5 Flash, Pro) for text and vision
- Automatic model detection and adapter selection

✅ **Tool Calling**
- Full tool/function calling support for GPT models
- Proper warnings for models that don't support tools
- Workaround for AI SDK v6 beta.93 schema serialization bug

✅ **Streaming Support**
- Text streaming for all models
- Real-time response generation

✅ **Vision Support**
- Image understanding with Gemini models
- Base64 encoded image support

✅ **Production Ready**
- Type-safe TypeScript implementation
- Comprehensive error handling
- Token usage tracking
- Request/response debugging

## Installation

```bash
pnpm install
```

## Quick Start

### Basic Setup

```typescript
import { createFoundry } from '@northslopetech/foundry-ai-sdk';
import { generateText } from 'ai';

const foundry = createFoundry({
  foundryToken: process.env.FOUNDRY_TOKEN!,
  baseURL: process.env.FOUNDRY_BASE_URL!,
});

// Simple text generation
const result = await generateText({
  model: foundry('Gemini_2_0_Flash'),
  prompt: 'What is 2+2?',
});

console.log(result.text);
```

### Tool Calling with GPT Models

**Important**: Due to a bug in AI SDK v6 beta.93, tool schemas must be passed via `providerOptions.foundry.parameters`:

```typescript
import { generateText } from 'ai';
import { createFoundry } from '@northslopetech/foundry-ai-sdk';

const foundry = createFoundry({
  foundryToken: process.env.FOUNDRY_TOKEN!,
  baseURL: process.env.FOUNDRY_BASE_URL!,
});

// Define your tool schema
const weatherToolSchema = {
  type: 'object' as const,
  properties: {
    location: {
      type: 'string' as const,
      description: 'The city and state, e.g. San Francisco, CA',
    },
    unit: {
      type: 'string' as const,
      enum: ['celsius', 'fahrenheit'],
    },
  },
  required: ['location'],
};

// WORKAROUND: Pass schema in both parameters AND providerOptions
const weatherTool = {
  description: 'Get the current weather in a given location',
  parameters: weatherToolSchema,
  providerOptions: {
    foundry: {
      parameters: weatherToolSchema, // Required for AI SDK beta.93
    },
  },
  execute: async ({ location, unit }) => {
    // Your tool implementation
    return {
      location,
      temperature: unit === 'celsius' ? 18 : 65,
      unit: unit || 'fahrenheit',
      conditions: 'Partly cloudy',
    };
  },
};

const result = await generateText({
  model: foundry('GPT_5'),
  prompt: 'What is the weather in San Francisco?',
  tools: { get_weather: weatherTool },
  maxTokens: 500,
});

console.log(result.text);
```

### Streaming

```typescript
import { streamText } from 'ai';

const result = streamText({
  model: foundry('Gemini_2_0_Flash'),
  prompt: 'Write a short poem about AI',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}
```

### Vision (Image Understanding)

```typescript
import { generateText } from 'ai';
import * as fs from 'fs';

const imageBuffer = fs.readFileSync('image.png');
const base64Image = imageBuffer.toString('base64');

const result = await generateText({
  model: foundry('Gemini_2_0_Flash'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is in this image?' },
        {
          type: 'image',
          image: base64Image,
          mimeType: 'image/png',
        },
      ],
    },
  ],
});

console.log(result.text);
```

## Supported Models

### GPT Models (with Tool Calling)
- `GPT_4`
- `GPT_5`
- Full tool/function calling support
- Vision support via `gptChatWithVision`

### Gemini Models (Text & Vision)
- `Gemini_2_0_Flash`
- `Gemini_2_5_Flash`
- `Gemini_2_5_Pro`
- **Note**: Tool calling not supported by Foundry's Gemini implementation

### Model Detection

The provider automatically detects the model family and applies the appropriate adapter:

```typescript
// Automatically uses GPT adapter with tool support
const gptModel = foundry('GPT_5');

// Automatically uses generic adapter (no tools)
const geminiModel = foundry('Gemini_2_0_Flash');
```

## Configuration

### Environment Variables

Create a `.env` file:

```bash
FOUNDRY_TOKEN=your_foundry_token
FOUNDRY_BASE_URL=https://yourinstance.palantirfoundry.com
```

### Provider Options

```typescript
const foundry = createFoundry({
  foundryToken: 'your-token',           // Required
  baseURL: 'https://...',               // Required
  headers: {                            // Optional
    'Custom-Header': 'value',
  },
});
```

### Model Settings

```typescript
const model = foundry('GPT_5', {
  temperature: 0.7,    // Optional
  maxTokens: 1000,     // Optional
});
```

## Debugging

Enable debug logging to see request/response details:

```bash
DEBUG_FOUNDRY=true pnpm tsx your-script.ts
```

This will log:
- Tool schemas being sent
- Request bodies
- Response parsing
- Error details

## Known Issues & Workarounds

### AI SDK v6 Beta.93 Tool Schema Bug

**Issue**: The AI SDK doesn't properly serialize tool schemas, sending empty `inputSchema` to providers.

**Workaround**: Pass schemas via `providerOptions.foundry.parameters`:

```typescript
const tool = {
  description: '...',
  parameters: schema,
  providerOptions: {
    foundry: { parameters: schema }, // Add this
  },
  execute: async (args) => { ... },
};
```

### Foundry GPT Quirks

1. **`tool_choice` parameter**: Foundry's GPT endpoint rejects the `tool_choice` parameter. The provider automatically omits it.

2. **`temperature` parameter**: Some GPT configurations reject temperature. The provider omits it for GPT requests.

3. **Response format**: Foundry wraps tool calls in a `toolCall` object. The provider handles both standard OpenAI and Foundry formats.

## Architecture

```
┌─────────────────────────────────────────────┐
│           Vercel AI SDK (v6 beta)           │
│         (generateText, streamText)          │
└────────────────┬────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────┐
│        FoundryLanguageModel                 │
│     (implements LanguageModelV2)            │
│                                             │
│  ┌─────────────────────────────────────┐   │
│  │  Model Detection                    │   │
│  │  (detectModel)                      │   │
│  └─────────────────────────────────────┘   │
│                 │                           │
│                 ▼                           │
│  ┌──────────────┴──────────────┐           │
│  │                              │           │
│  ▼                              ▼           │
│ GPT Adapter              Generic Adapter    │
│ ├─ Request Adapter       (Gemini, etc)      │
│ └─ Response Parser       - genericVision    │
│    - Tool calling           - No tools      │
│    - gptChat format         - Warnings      │
└─────────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────────┐
│     Foundry Language Model Service          │
│   /llm/v3/completion/{modelId}              │
│   /llm/v3/completion/{modelId}/stream       │
└─────────────────────────────────────────────┘
```

## API Reference

### `createFoundry(options)`

Creates a Foundry provider instance.

**Parameters:**
- `foundryToken`: Your Foundry authentication token
- `baseURL`: Your Foundry instance URL
- `headers?`: Optional additional headers

**Returns:** `FoundryProvider`

### Model Interface

```typescript
interface FoundryLanguageModel extends LanguageModelV2 {
  provider: string;           // 'foundry'
  modelId: string;           // e.g., 'GPT_5'
  settings: {
    temperature?: number;
    maxTokens?: number;
  };

  doGenerate(options): Promise<GenerateResult>;
  doStream(options): Promise<StreamResult>;
}
```

## Examples

See the `examples/` directory for complete working examples:

- `basic-text-completion.ts` - Simple text generation
- `gpt-tool-calling.ts` - Tool calling with GPT models
- `vision-completion.ts` - Image understanding
- `test-all-features.ts` - Comprehensive feature test
- `complete-workflow-test.ts` - Full multi-step workflow

Run examples:

```bash
pnpm tsx examples/basic-text-completion.ts
pnpm tsx examples/gpt-tool-calling.ts
```

## Testing

```bash
# Run all tests
pnpm test:all

# Run specific tests
pnpm test:basic
pnpm test:vision
pnpm test:gpt-tools

# Verify API connectivity
pnpm verify
pnpm verify:gpt
```

## Development

### Project Structure

```
src/
├── index.ts                      # Main exports
├── foundry-provider.ts           # Provider factory
├── foundry-language-model.ts     # Core model implementation
├── foundry-types.ts              # Foundry API types
├── foundry-settings.ts           # Configuration types
├── foundry-prompt-converter.ts   # Message format conversion
├── model-detector.ts             # Model family detection
└── adapters/
    ├── adapter-interface.ts      # Adapter contracts
    ├── gpt-request-adapter.ts    # GPT request conversion
    ├── gpt-response-parser.ts    # GPT response parsing
    └── generic-chat-request-adapter.ts

examples/
├── basic-text-completion.ts
├── gpt-tool-calling.ts
├── vision-completion.ts
└── test-all-features.ts

scripts/
├── verify-chat-completion.ts
├── verify-vision-completion.ts
└── verify-gpt-tool-calling.ts
```

### Adding New Model Adapters

1. Create request adapter implementing `RequestAdapter`
2. Create response parser implementing `ResponseParser`
3. Add model detection logic in `model-detector.ts`
4. Register adapter in `FoundryLanguageModel` constructor

Example:

```typescript
if (this.modelInfo.family === 'claude') {
  this.adapter = new ClaudeRequestAdapter();
  this.parser = new ClaudeResponseParser();
}
```

## Contributing

Contributions welcome! Please ensure:

1. All tests pass: `pnpm test`
2. Code is formatted: `pnpm format`
3. Examples work: `pnpm test:all`

## License

MIT

## Support

For issues related to:
- **This provider**: Open an issue on GitHub
- **Foundry LMS**: Contact Palantir support
- **AI SDK**: Check [Vercel AI SDK docs](https://sdk.vercel.ai/)

## Changelog

### v0.1.0 (Current)

- ✅ Initial release
- ✅ GPT model support with tool calling
- ✅ Gemini model support (text & vision)
- ✅ Streaming support
- ✅ Vision support
- ✅ Comprehensive error handling
- ✅ AI SDK v6 beta compatibility
- ⚠️ Workaround for AI SDK beta.93 tool schema bug

## Acknowledgments

Built for Northslope Technologies to enable AI-powered applications on Palantir Foundry.