npm package: @nebulaos/llm-gateway
Description: NebulaOS LLM Gateway provider - OpenAI-compatible chat completions
Latest version: 0.1.8

# @nebulaos/llm-gateway

NebulaOS LLM Gateway provider for the NebulaOS SDK. Provides OpenAI-compatible chat completions through pre-configured routes with automatic fallback, cost tracking, and access control.

## Installation

```bash
npm install @nebulaos/llm-gateway
```

## Usage

```typescript
import { LLMGateway } from "@nebulaos/llm-gateway";
import { Agent, BufferMemory } from "@nebulaos/core";

const agent = new Agent({
  id: "assistente",
  name: "assistente",
  model: new LLMGateway({
    apiKey: process.env.NEBULAOS_API_KEY,
    baseUrl: "https://your-nebula-instance.com",
    model: "assistente", // Route alias configured in NebulaOS
    options: {
      reasoning_effort: "high",  // Enable deep reasoning (Gemini)
      temperature: 0.7,
    },
  }),
  memory: new BufferMemory(),
  instructions: "You are a helpful assistant.",
});

// Use the agent
const result = await agent.execute("Analyze this complex problem step by step");
console.log(result.content);
```

## Configuration

| Option | Type | Required | Description |
|--------|------|----------|-------------|
| `apiKey` | `string` | Yes | API key from NebulaOS |
| `baseUrl` | `string` | No | NebulaOS instance URL (defaults to localhost:4100) |
| `model` | `string` | Yes | Route alias configured in NebulaOS |
| `logLevel` | `string` | No | Logger verbosity ("debug", "info", "warn", "error", "none") |
| `options` | `object` | No | Default model options for every call (see below) |

## Model Options

Use `options` to configure model-specific parameters that apply to every call:

```typescript
const model = new LLMGateway({
  apiKey: "your-api-key",
  model: "minha-rota",
  options: {
    temperature: 0.7,
    reasoning_effort: "high",  // Gemini 2.5+/3+ via LiteLLM
    topK: 40,
  }
});
```

### Common Options

| Option | Type | Description |
|--------|------|-------------|
| `temperature` | `number` | Sampling temperature (0-2) |
| `topP` | `number` | Nucleus sampling |
| `topK` | `number` | Top-K sampling (Gemini) |
| `maxTokens` | `number` | Max output tokens |

### Provider-Specific Options

These are passed directly to the provider via LiteLLM:

**Gemini (via LiteLLM)**:
| Option | Values | Description |
|--------|--------|-------------|
| `reasoning_effort` | `"low"`, `"medium"`, `"high"` | Controls thinking depth (Gemini 2.5+/3+) |
| `thinking` | `{ type: "enabled", budget_tokens: N }` | Fine-grained thinking control |

**OpenAI**:
| Option | Type | Description |
|--------|------|-------------|
| `frequency_penalty` | `number` | Reduce repetition (-2 to 2) |
| `presence_penalty` | `number` | Encourage new topics (-2 to 2) |

Any extra options are passed through to the underlying provider.

## Routes

Routes are configured in the NebulaOS UI and provide:

- **Automatic fallback**: If primary model fails, automatically tries backup models
- **Cost tracking**: All usage is automatically tracked and billed
- **Access control**: API keys can be restricted to specific routes
- **Rate limiting**: Configurable limits per route and user

## Supported Features

- ✅ Text chat completions
- ✅ Streaming responses
- ✅ Tool/function calling
- ✅ JSON mode
- ✅ Multimodal (images)
- ✅ Automatic cost tracking
- ✅ Route-based access control

## Error Handling

The provider will throw errors for:
- Invalid API keys
- Routes not found or inactive
- Access denied to routes
- Network issues with NebulaOS

All errors include appropriate HTTP status codes and messages.