npm package: @dwmkerr/mock-llm
Description: Simple OpenAI compatible Mock API server. Useful for deterministic testing of AI applications.
Author: Dave Kerr
Latest version: 0.1.28

# mock-llm

[![npm version](https://badge.fury.io/js/@dwmkerr%2Fmock-llm.svg)](https://badge.fury.io/js/@dwmkerr%2Fmock-llm)
[![codecov](https://codecov.io/gh/dwmkerr/mock-llm/graph/badge.svg?token=0GvcoTYVY4)](https://codecov.io/gh/dwmkerr/mock-llm)
[![All Contributors](https://img.shields.io/github/all-contributors/dwmkerr/mock-llm?color=ee8449&style=flat-square)](#contributors)

Simple OpenAI compatible Mock API server. Useful for deterministic testing of AI applications.

## Introduction

Creating integration tests for AI applications that rely on LLMs can be challenging due to costs, the complexity of response structures and the non-deterministic nature of LLMs. Mock LLM runs as a simple 'echo' server that responses to a user message.

The server can be configured to provide different responses based on the input, which can be useful for testing error scenarios, different payloads, etc. It is currently designed to mock the [OpenAI Completions API](https://platform.openai.com/docs/api-reference/completions) but could be extended to mock the list models APIs, responses APIs, A2A apis and so on in the future.

<!-- vim-markdown-toc GFM -->

- [Quickstart](#quickstart)
- [Configuration](#configuration)
    - [Customising Responses](#customising-responses)
    - [Loading Configuration Files](#loading-configuration-files)
    - [Updating Configuration](#updating-configuration)
    - [Health & Readiness Checks](#health--readiness-checks)
    - [Template Variables](#template-variables)
    - [Sequential Responses](#sequential-responses)
    - [Streaming Configuration](#streaming-configuration)
- [MCP (Model Context Protocol) Mocking](#mcp-model-context-protocol-mocking)
- [A2A (Agent to Agent Protocol) Mocking](#a2a-agent-to-agent-protocol-mocking)
- [Deploying to Kubernetes with Helm](#deploying-to-kubernetes-with-helm)
- [Examples](#examples)
- [Developer Guide](#developer-guide)
- [Samples](#samples)
- [Contributors](#contributors)

<!-- vim-markdown-toc -->

## Quickstart

Install and run:

```bash
npm install -g mock-llm
mock-llm
```

Mock-LLM runs on port 6556 (which is dial-pad code 6556, to avoid conflicts with
common ports).

Or use Docker:

```bash
docker run -p 6556:6556 ghcr.io/dwmkerr/mock-llm
```

Or [use Helm](#deploying-to-kubernetes-with-helm) for Kubernetes deployments.

Test with curl. The default rule for incoming requests is to reply with the user's exact message:

```bash
curl -X POST http://localhost:6556/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

Response:

```json
{
  "id": "chatcmpl-1234567890",
  "object": "chat.completion",
  "model": "gpt-4",
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "Hello"
    },
    "finish_reason": "stop"
  }]
}
```

Mock LLM also has basic support for the [A2A (Agent-to-Agent) protocol](docs/a2a.md) for testing agent messages, task, and asynchronous operations.

## Configuration

Responses are configured using a `yaml` file loaded from `mock-llm.yaml` in the current working directory. Rules are evaluated in order - last match wins.

The default configuration echoes the last user message and provides a `/v1/models` endpoint:

```yaml
rules:
  # Default echo rule
  - path: "/v1/chat/completions"
    method: "POST"
    # The JMESPath expression '@' always matches.
    match: "@"
    response:
      status: 200
      content: |
        {
          "id": "chatcmpl-{{timestamp}}",
          "object": "chat.completion",
          "model": "{{jmes request body.model}}",
          "choices": [{
            "message": {
              "role": "assistant",
              "content": "{{jmes request body.messages[-1].content}}"
            },
            "finish_reason": "stop"
          }]
        }

  # List models endpoint
  - path: "/v1/models"
    method: "GET"
    response:
      status: 200
      content: |
        {
          "object": "list",
          "data": [{"id": "gpt-5.2", "object": "model", "owned_by": "openai"}]
        }
```

Each rule supports these fields:
- `path` - Regular expression to match the request path
- `method` - HTTP method to match (GET, POST, etc). If not specified, matches all methods
- `match` - JMESPath expression to match request content (optional)
- `sequence` - Match by request order (optional)
- `response` - The response to return (status and content)

### Customising Responses

[JMESPath](https://jmespath.org/) is a query language for JSON used to match incoming requests and extract values for responses.

This returns a fixed message for `hello` and simulates a `401` error for `error-401`, and simulates `v1/models`:

```yaml
rules:
  # Fixed message when input contains 'hello':
  - path: "/v1/chat/completions"
    match: "contains(body.messages[-1].content, 'hello')"
    response:
      status: 200
      content: |
        {
          "choices": [{
            "message": {
              "role": "assistant",
              "content": "Hi there! How can I help you today?"
            },
            "finish_reason": "stop"
          }]
        }
  # Realistic OpenAI 401 if the input contains `error-401`:
  - path: "/v1/chat/completions"
    match: "contains(body.messages[-1].content, 'error-401')"
    response:
      status: 401
      content: |
        {
          "error": {
            "message": "Incorrect API key provided.",
            "type": "invalid_request_error",
            "param": null,
            "code": "invalid_api_key"
          }
        }

  # List models endpoint
  - path: "/v1/models"
    # The JMESPath expression '@' always matches.
    match: "@"
    response:
      status: 200
      # Return a set of models.
      content: |
        {
          "data": [
            {"id": "gpt-4", "object": "model"},
            {"id": "gpt-3.5-turbo", "object": "model"}
          ]
        }
```

### Loading Configuration Files

The `--config` parameter can be used for a non-default location:

```bash
# Use the '--config' parameter directly...
mock-llm --config /tmp/myconfig.yaml

# ...mount a config file from the working directory for mock-llm in docker.
docker run -v $(pwd)/mock-llm.yaml:/app/mock-llm.yaml -p 6556:6556 ghcr.io/dwmkerr/mock-llm
```

### Updating Configuration

Configuration can be updated at runtime via the `/config` endpoint: `GET` returns current config (JSON by default, YAML with `Accept: application/x-yaml`), `POST` replaces it, `PATCH` merges updates, `DELETE` resets to default. Both `POST` and `PATCH` accept JSON (`Content-Type: application/json`) or YAML (`Content-Type: application/x-yaml`).

### Health & Readiness Checks

```bash
curl http://localhost:6556/health
# {"status":"healthy"}

curl http://localhost:6556/ready
# {"status":"ready"}
```

### Template Variables

Available in response content templates:

- `{{jmes request <query>}}` - Query the request object using [JMESPath](https://jmespath.org/):
  - `request.body` - Request body (e.g., `body.model`, `body.messages[-1].content`)
  - `request.headers` - HTTP headers, lowercase (e.g., `headers.authorization`)
  - `request.method` - HTTP method (e.g., `POST`)
  - `request.path` - Request path (e.g., `/v1/chat/completions`)
  - `request.query` - Query parameters (e.g., `query.apikey`)
- `{{timestamp}}` - Current time in milliseconds

Objects and arrays are automatically JSON-stringified. Primitives are returned as-is.

```yaml
"model": "{{jmes request body.model}}"              // "gpt-4"
"message": {{jmes request body.messages[0]}}        // {"role":"system","content":"..."}
"auth": "{{jmes request headers.authorization}}"    // "Bearer sk-..."
"apikey": "{{jmes request query.apikey}}"           // "test-123"
```

### Sequential Responses

For testing multi-turn interactions like tool calling, use `sequence` to return different responses based on request order:

```yaml
rules:
  # First request: trigger tool call
  - path: "/v1/chat/completions"
    sequence: 0
    response:
      status: 200
      content: '{"choices":[{"message":{"tool_calls":[{"function":{"name":"get_weather"}}]},"finish_reason":"tool_calls"}]}'

  # Second request: return final answer
  - path: "/v1/chat/completions"
    sequence: 1
    response:
      status: 200
      content: '{"choices":[{"message":{"content":"The weather is 72°F"},"finish_reason":"stop"}]}'
```

Rules can use `match`, `sequence`, both, or neither:

| `match` | `sequence` | Behavior |
|---------|------------|----------|
| No | No | Matches all requests (catch-all) |
| Yes | No | Content-based matching only |
| No | Yes | Order-based matching only |
| Yes | Yes | Must satisfy both conditions |

Sequence counters are tracked per path and reset via `DELETE /config`. The counter only increments when the winning rule has a `sequence` property, so fallback rules (without `sequence`) can handle liveness probes without consuming sequence numbers.

### Streaming Configuration

Mock-LLM supports streaming responses when clients send `stream: true` in their requests. Streaming behavior is configured globally:

```yaml
streaming:
  chunkSize: 50         # characters per chunk (default: 50)
  chunkIntervalMs: 50   # milliseconds between chunks (default: 50)

rules:
  - path: "/v1/chat/completions"
    match: "@"
    # etc...
```

When clients request streaming, Mock-LLM returns Server-Sent Events (SSE) with `Content-Type: text/event-stream`:

```javascript
const stream = await client.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello' }],
  stream: true  // Enables streaming
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```

This enables deterministic testing of streaming protocol responses. Errors conditions can also be tested - error responses are sent as per the [OpenAI Streaming Specification](https://platform.openai.com/docs/api-reference/chat-streaming/streaming).

## MCP (Model Context Protocol) Mocking

Mock-LLM exposes MCP servers and tools which support testing the MCP protocol, details are in the [MCP Documentation](docs/mcp.md).

## A2A (Agent to Agent Protocol) Mocking

Mock-LLM exposes A2A servers and tools which support testing the A2A protocol, details are in the [A2A Documentation](docs/a2a.md).

## Deploying to Kubernetes with Helm

```bash
# Install from OCI registry
helm install mock-llm oci://ghcr.io/dwmkerr/charts/mock-llm --version 0.1.8

# Install with Ark resources enabled
# Requires Ark to be installed: https://github.com/mckinsey/agents-at-scale-ark
helm install mock-llm oci://ghcr.io/dwmkerr/charts/mock-llm --version 0.1.8 \
  --set ark.model.enabled=true \
  --set ark.a2a.enabled=true \
  --set ark.mcp.enabled=true

# Verify deployment
kubectl get deployment mock-llm
kubectl get service mock-llm

# Port forward and test
kubectl port-forward svc/mock-llm 6556:6556 &
curl -X POST http://localhost:6556/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}'
```

Custom configuration via values.yaml:

```yaml

# Optional additional mock-llm configuration.
config:
  rules:
    - path: "/v1/chat/completions"
      match: "contains(messages[-1].content, 'hello')"
      response:
        status: 200
        content: |
          {
            "choices": [{
              "message": {
                "role": "assistant",
                "content": "Hi there!"
              },
              "finish_reason": "stop"
            }]
          }

# Or use existing ConfigMap (must contain key 'mock-llm.yaml')
# existingConfigMap: "my-custom-config"
```

See the [full Helm documentation](docs/helm.md) for advanced configuration, Ark integration, and more.

## Examples

Any OpenAI API compatible SDKs can be used with Mock LLM. For Node.js:

```javascript
const OpenAI = require('openai');

const client = new OpenAI({
  apiKey: 'mock-key',
  baseURL: 'http://localhost:6556/v1'
});

const response = await client.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello' }]
});

console.log(response.choices[0].message.content);
// "Hello"
```

And for Python:

```python
from openai import OpenAI

client = OpenAI(
    api_key='mock-key',
    base_url='http://localhost:6556/v1'
)

response = client.chat.completions.create(
    model='gpt-4',
    messages=[{'role': 'user', 'content': 'Hello'}]
)

print(response.choices[0].message.content)
# "Hello"
```

## Developer Guide

Install dependencies and start with live-reload:

```bash
npm install
npm run dev
```

Lint or run tests:

```bash
npm run lint
npm run test
```

Test and inspect the MCP Server running locally:

```bash
npm run local:inspect
```

## Samples

Each sample below is in the form of an extremely minimal script that shows:

1. How to configure mock-llm for a specific scenario
2. How to run the scenario
3. How to validate the results

These can be a reference for your own tests. Each sample is also run as part of the project's build pipeline.

| Sample | Description |
|--------|-------------|
| [01-echo-message.sh](samples/01-echo-message.sh) | Assert a response from an LLM. |
| [02-error-401.sh](samples/02-error-401.sh) | Verify error handling scenario. |
| [03-system-message-in-conversation.sh](samples/03-system-message-in-conversation.sh) | Test system message handling in conversations. |
| [04-headers-validation.sh](samples/04-headers-validation.sh) | Test custom HTTP header validation. |
| [05-a2a-countdown-agent.sh](samples/05-a2a-countdown-agent.sh) | Test A2A blocking task operations. |
| [06-a2a-echo-agent.sh](samples/06-a2a-echo-agent.sh) | Test A2A message handling. |
| [07-a2a-message-context.sh](samples/07-a2a-message-context.sh) | Test A2A message context and history. |
| [08-mcp-echo-tool.sh](samples/08-mcp-echo-tool.sh) | Test MCP tool invocation. |
| [09-token-usage.sh](samples/09-token-usage.sh) | Test token usage tracking. |
| [10-mcp-inspect-headers.sh](samples/10-mcp-inspect-headers.sh) | Test MCP header inspection. |
| [11-sequential-tool-calling.sh](samples/11-sequential-tool-calling.sh) | Test sequential responses for tool-calling flows. |
| [12-list-models.sh](samples/12-list-models.sh) | Test GET /v1/models endpoint. |

Each sample below is a link to a real-world deterministic integration test in [Ark](https://github.com/mckinsey/agents-at-scale-ark) that uses `mock-llm` features. These tests can be used as a reference for your own tests.

| Test | Description |
|------|-------------|
| [agent-default-model](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/agent-default-model) | Basic LLM query and response. |
| [model-custom-headers](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/model-custom-headers) | Passing custom headers to models. |
| [query-parameter-ref](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/query-parameter-ref) | Dynamic prompt resolution from ConfigMaps and Secrets. |
| [query-token-usage](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/query-token-usage) | Token usage tracking and reporting. |
| [a2a-agent-discovery](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/a2a-agent-discovery) | A2A agent discovery and server readiness. |
| [a2a-message-query](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/a2a-message-query) | A2A message handling. |
| [a2a-blocking-task-completed](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/a2a-blocking-task-completed) | A2A blocking task successful completion. |
| [a2a-blocking-task-failed](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/a2a-blocking-task-failed) | A2A blocking task error handling. |
| [mcp-discovery](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/mcp-discovery) | MCP server and tool discovery. |
| [mcp-header-propagation (PR #311)](https://github.com/mckinsey/agents-at-scale-ark/pull/311) | MCP header propagation from Agents and Queries. |
| [agent-tools](https://github.com/mckinsey/agents-at-scale-ark/tree/main/tests/agent-tools) | Sequential responses for tool-calling agents. |

## Contributors

Thanks to ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
  <tbody>
    <tr>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/dwmkerr"><img src="https://avatars.githubusercontent.com/u/1926984?v=4?s=100" width="100px;" alt="Dave Kerr"/><br /><sub><b>Dave Kerr</b></sub></a><br /><a href="https://github.com/dwmkerr/mock-llm/commits?author=dwmkerr" title="Code">💻</a> <a href="https://github.com/dwmkerr/mock-llm/commits?author=dwmkerr" title="Documentation">📖</a> <a href="#infra-dwmkerr" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="#maintenance-dwmkerr" title="Maintenance">🚧</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/lucaromagnoli"><img src="https://avatars.githubusercontent.com/u/38782977?v=4?s=100" width="100px;" alt="Luca Romagnoli"/><br /><sub><b>Luca Romagnoli</b></sub></a><br /><a href="https://github.com/dwmkerr/mock-llm/commits?author=lucaromagnoli" title="Code">💻</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/daniele-marostica"><img src="https://avatars.githubusercontent.com/u/238710818?v=4?s=100" width="100px;" alt="Daniele"/><br /><sub><b>Daniele</b></sub></a><br /><a href="https://github.com/dwmkerr/mock-llm/commits?author=daniele-marostica" title="Code">💻</a></td>
      <td align="center" valign="top" width="14.28%"><a href="https://github.com/mcowger"><img src="https://avatars.githubusercontent.com/u/1929548?v=4?s=100" width="100px;" alt="Matt Cowger"/><br /><sub><b>Matt Cowger</b></sub></a><br /><a href="https://github.com/dwmkerr/mock-llm/commits?author=mcowger" title="Code">💻</a></td>
    </tr>
  </tbody>
</table>

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

<!-- ALL-CONTRIBUTORS-LIST:END -->

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome.