npm package: @semantictools/ai-toolbox
Description: A flexible tool manager for managing and executing AI function calling toolboxes
Author: Dusty Wilhelm Murray / Semantic Tools
Latest version: 0.1.8

# @semantictools/ai-toolbox

A flexible tool manager for managing and executing AI function calling toolboxes. This package provides a centralized system for registering, organizing, and executing functions that can be used with AI models supporting function calling (like Anthropic Claude, OpenAI GPT, etc.).

**Note:** This package uses a standardized interface format. See the [Interface Standard](#interface-standard) section below for detailed specifications.

## Features

- **Toolbox Registration**: Register multiple toolboxes with their own sets of functions
- **Hierarchical Organization**: Support for root toolboxes and namespaced function calls
- **Function Discovery**: List and find functions across all registered toolboxes
- **Flexible Calling**: Call functions using dot notation (e.g., `toolbox.function`)
- **Built-in Translators**: Production-ready translator included (`atfunc`) for AI text-based function calling
- **Custom Translators**: Support for custom function interface translators with configurable case sensitivity
- **Response Parsing**: Extract and execute function calls from AI model text responses
- **Control Functions**: Built-in introspection functions (domain: `tools`) for dynamic function discovery
- **Interface Validation**: Automatic validation of function interfaces during registration
- **Parameter Validation**: Runtime validation of required parameters with clear error messages
- **Statistics Tracking**: Monitor tool and function call counts
- **Result Truncation**: Configurable truncation for long function outputs
- **Standardized Interface**: Well-defined interface format for consistent toolbox development

## Installation

```bash
npm install @semantictools/ai-toolbox
```

## Quick Start

```javascript
import {
  register,
  setRootToolBox,
  setFunctionTranslator,
  TRANSLATOR_ATFUNC,
  call,
  getFunctions,
  executeResponseFunctions
} from '@semantictools/ai-toolbox';

// 1. Set up the built-in translator (do this first!)
setFunctionTranslator(TRANSLATOR_ATFUNC);

// 2. Create your toolbox
const myToolBox = {
  getName: () => 'mytoolbox',
  getFunction: (fname) => {
    if (fname === 'greet') {
      return {
        f: async (args) => `Hello, ${args.name}!`,
        interface: {
          domain: 'mytoolbox',    // Must match getName()
          name: 'greet',          // Lowercase, snake_case recommended
          type: 'function',       // 'function' or 'api-wrapper'
          description: 'Greet a person by name',
          params: {               // Use 'params', not 'parameters'
            name: {
              type: 'string',
              isRequired: true,   // Use 'isRequired', not 'required'
              description: 'Person\'s name'
            }
          },
          returns: 'string'
        }
      };
    }
    return null;
  },
  functions: { greet: true }
};

// 3. Register the toolbox
register(myToolBox);
setRootToolBox('mytoolbox');

// 4. Call functions
const result = await call('greet', { name: 'World' });
console.log(result); // "Hello, World!"
```

## API Reference

### Core Functions

#### `register(toolBox)`
Register a toolbox with the tool manager.

**Parameters:**
- `toolBox`: Object with `getName()`, `getFunction(fname)`, and `functions` property

#### `setRootToolBox(toolBoxName)`
Set the root toolbox (functions can be called without namespace prefix).

**Parameters:**
- `toolBoxName`: String name of the toolbox

#### `call(toolPath, args, truncateLength)`
Call a function by its path.

**Parameters:**
- `toolPath`: String in format `'function'` or `'toolbox.function'`
- `args`: Object with function arguments
- `truncateLength`: Optional number for result truncation (default: 5000)

**Returns:** Function result

#### `getFunctions(toolBoxName)`
Get list of all functions or functions in a specific toolbox.

**Parameters:**
- `toolBoxName`: Optional string to filter by toolbox

**Returns:** Array of function names

#### `findFunction(toolPath)`
Find a function by its path.

**Parameters:**
- `toolPath`: String in format `'function'` or `'toolbox.function'`

**Returns:** Function object with `f` (function) and `interface` properties

### Advanced Features

#### `setFunctionTranslator(converterObject)`
Set a custom function interface converter for AI model compatibility.

**Parameters:**
- `converterObject`: Translator object, translator name string, or translator constant (e.g., `TRANSLATOR_ATFUNC`)

**Examples:**
```javascript
// Option 1: Use translator constant (recommended)
import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';
setFunctionTranslator(TRANSLATOR_ATFUNC);

// Option 2: Use translator name string
setFunctionTranslator('atfunc');

// Option 3: Load explicitly with getTranslator
const translator = getTranslator('atfunc');
setFunctionTranslator(translator);

// Option 4: Use custom translator object
import * as myTranslator from './my-translator.mjs';
setFunctionTranslator(myTranslator);
```

#### `getTranslator(name)`
Get a built-in standard translator by name.

**Parameters:**
- `name`: String name of the translator (e.g., `'atfunc'`)

**Returns:** Translator object with `generateFunctionIfText`, `getFunctionCallsFromText`, and `formatFunctionCallResults`

**Example:**
```javascript
import { getTranslator, setFunctionTranslator } from '@semantictools/ai-toolbox';

const translator = getTranslator('atfunc');
setFunctionTranslator(translator);
```

#### `getStandardTranslatorList()`
Get list of all available built-in translators.

**Returns:** Array of translator names

**Example:**
```javascript
import { getStandardTranslatorList } from '@semantictools/ai-toolbox';

console.log(getStandardTranslatorList()); // ['atfunc']
```

### Translator Constants

The package exports constants for easy access to built-in translators:

- **`TRANSLATOR_ATFUNC`** - The atfunc production translator

**Example:**
```javascript
import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';

// Most convenient way to set a built-in translator
setFunctionTranslator(TRANSLATOR_ATFUNC);
```

### Control Functions

The toolmanager provides built-in control functions (domain: `tools`) for introspection and management. These are **registered by default** and can be called like any other function.

#### Available Control Functions

##### `tools.get_toolboxes()`
Get list of all registered toolboxes.

**Returns:** Array of toolbox names

##### `tools.get_functions(toolbox?)`
Get all functions, optionally filtered by toolbox.

**Parameters:**
- `toolbox` (optional): String - Filter by toolbox name

**Returns:** Array of function names

##### `tools.get_function_interface(function_name, separator?)`
Get the interface definition of a specific function as text.

**Parameters:**
- `function_name` (required): String - Full function name (e.g., "weather.get_temp")
- `separator` (optional): Boolean - Add visual separator

**Returns:** String with function interface in text format

##### `tools.set_truncate_length(length)`
Set the output truncation length for function results.

**Parameters:**
- `length` (required): Number - Truncation length in characters

**Returns:** Confirmation string

#### Managing Control Functions

```javascript
import {
  registerControlFunctions,
  unregisterControlFunctions,
  registerControlFunction,
  unregisterControlFunction
} from '@semantictools/ai-toolbox';

// Unregister all control functions
unregisterControlFunctions();

// Re-register all control functions
registerControlFunctions();

// Register individual control function
registerControlFunction('get_toolboxes');

// Unregister individual control function
unregisterControlFunction('get_toolboxes');
```

#### Usage Examples

**Direct calls:**
```javascript
const toolboxes = await call('tools.get_toolboxes');
const allFunctions = await call('tools.get_functions');
const weatherFunctions = await call('tools.get_functions', { toolbox: { value: 'weather' } });
```

**AI text-based calls:**
```javascript
const aiResponse = `
@tools.get_toolboxes()
@tools.get_functions("weather")
`;
const result = await executeResponseFunctions(aiResponse);
```

**Use cases:**
- Let AI discover available functions dynamically
- Allow AI to inspect function interfaces
- Enable AI to adjust truncation settings
- Debug and introspection during development

## Validation

The toolmanager includes comprehensive validation to catch errors early and provide clear error messages.

### Interface Validation

Function interfaces are automatically validated when toolboxes are registered. This catches configuration errors before runtime.

**What is validated:**
- Required fields: `domain`, `name`, `type`, `description`, `returns`
- `type` must be `'function'` or `'api-wrapper'`
- `domain` must match toolbox `getName()`
- Parameter structure: each param must have `type`, `isRequired`, `description`
- For API wrappers: validates `path_params`, `query_params` structure

**Example error:**
```javascript
const badToolbox = {
  getName: () => 'weather',
  getFunction: () => ({
    f: async () => 'result',
    interface: {
      domain: 'wrong',  // Doesn't match 'weather'
      name: 'get_temp',
      type: 'function',
      // ... other fields
    }
  }),
  functions: { get_temp: true }
};

register(badToolbox);
// Throws: "Validation failed for function 'get_temp' in toolbox 'weather':
//          Interface domain 'wrong' does not match toolbox name 'weather'"
```

### Parameter Validation

Required parameters are validated at runtime when functions are called, providing immediate feedback.

**What is validated:**
- All parameters with `isRequired: true` must be present in function calls
- Validation happens in both `call()` and `executeResponseFunctions()`
- Clear error messages identify which parameters are missing

**Example error:**
```javascript
// Interface requires 'location' parameter
await call('weather.get_temp');  // Missing location

// Throws: "Missing required parameter for function 'weather.get_temp': location"
```

**Handling validation errors in AI responses:**
```javascript
const aiResponse = '@weather.get_temp()';  // Missing required param
const result = await executeResponseFunctions(aiResponse);

// Result includes validation error:
// "Function call: weather.get_temp()
//  Returned: Error: Missing required parameter for function 'weather.get_temp': location"
```

### Benefits

✅ **Fail Fast** - Errors caught at registration/call time, not deep in execution
✅ **Clear Messages** - Know exactly what's wrong and where
✅ **Development Aid** - Catch typos and mistakes during development
✅ **Type Safety** - Ensure interfaces match expected structure
✅ **AI Safety** - Handle malformed AI function calls gracefully

#### `executeResponseFunctions(promptText)`
Parse and execute function calls from AI model response text.

**Parameters:**
- `promptText`: String containing function calls

**Returns:** Object with `functionsOutput` and `functionCalCount`

#### `getRootInterfaceAsText()`
Get all function interfaces as formatted text.

**Returns:** String with all function interface descriptions

#### `getStats()`
Get statistics on tool and function calls.

**Returns:** Object with `toolCalls` and `functionCalls` counts

#### `resetStats()`
Reset call statistics to zero.

## Toolbox Structure

A toolbox must implement:

```javascript
{
  getName: () => 'toolbox-name',
  getFunction: (functionName) => {
    return {
      f: async (args, truncateLength) => {
        // Function implementation
        return result;
      },
      interface: {
        domain: 'toolbox-name',    // Must match getName()
        name: 'function_name',     // Lowercase, use snake_case
        type: 'function',          // 'function' or 'api-wrapper'
        description: 'What the function does',
        params: {                  // Use 'params', not 'parameters'
          param1: {
            type: 'string',
            isRequired: true,      // Use 'isRequired', not 'required'
            description: 'Parameter description'
          }
        },
        returns: 'string' // or 'object', 'array', etc.
      }
    };
  },
  functions: {
    'function_name': true
    // ... list all available functions
  }
}
```

**Important:** Case sensitivity is determined by your translator. The `atfunc` translator is case-insensitive (flexible for AI), while custom translators can be case-sensitive or case-insensitive based on their needs. See the [Case Sensitivity](#case-sensitivity) section for details.

## Interface Standard

### Function Interface Format

All function interfaces must follow this standardized structure:

```javascript
{
  domain: 'toolbox-name',    // The toolbox name (must match getName())
  name: 'function_name',     // Function name (lowercase, snake_case recommended)
  type: 'function',          // Type: 'function' or 'api-wrapper'
  description: 'What the function does',
  params: {                  // Function parameters (note: 'params', not 'parameters')
    param_name: {
      type: 'string' | 'number' | 'boolean' | 'object',
      isRequired: true | false,  // Note: 'isRequired', not 'required'
      description: 'Parameter description'
    }
  },
  returns: 'string' | 'number' | 'object' | 'array'
}
```

#### Required Fields

- **domain**: Must match the toolbox name from `getName()`
- **name**: Function name (lowercase, preferably snake_case)
- **type**: Either `'function'` or `'api-wrapper'`
- **description**: Clear description of what the function does
- **params**: Object containing parameter definitions
- **returns**: The return type of the function

#### Naming Conventions

Naming conventions depend on your translator's case sensitivity:

**For case-insensitive translators (like atfunc):**
- Use any casing style (camelCase, snake_case, PascalCase)
- `getCurrentWeather`, `get_current_weather`, or `GetCurrentWeather` all work
- Recommendation: Use snake_case for consistency with the text format

**For case-sensitive translators:**
- Use exact case matching
- `getCurrentWeather` ≠ `getcurrentweather`
- Choose a consistent style (camelCase or snake_case)

**General recommendations:**
- **Function names**: snake_case (e.g., `get_current_weather`)
- **Domain/toolbox names**: lowercase or PascalCase
- **Parameter names**: snake_case for consistency

#### API Wrapper Type

For `type: 'api-wrapper'`, additional fields are supported:

```javascript
{
  domain: 'api',
  name: 'get_user',
  type: 'api-wrapper',
  description: 'Get user information',
  path_params: {
    user_id: {
      type: 'string',
      isRequired: true,
      description: 'User ID'
    }
  },
  query_params: {
    include_details: {
      type: 'boolean',
      isRequired: false,
      description: 'Include detailed information'
    }
  },
  request_body: {
    field_name: {
      type: 'string',
      required: true,
      description: 'Field description'
    }
  },
  response_type: 'application/json',
  returns: 'object'
}
```

### Function Translator Interface

A translator enables text-based function calling for AI model integration. To create a translator, implement these three functions:

#### 1. `generateFunctionIfText(funcInterface, useSeparator)`

Converts a function interface to text format for AI consumption.

**Parameters:**
- `funcInterface`: The function interface object
- `useSeparator`: Boolean to add visual separator (optional)

**Returns:** String representation of the function

**Example Output:**
```
@weather.get_current_weather( <*location>, <unit> )
Description: Get current weather information for a location
```

#### 2. `getFunctionCallsFromText(text, interfaces, rootToolBox)`

Parses text to extract function calls.

**Parameters:**
- `text`: String containing function calls
- `interfaces`: Array of all available function interfaces
- `rootToolBox`: String name of root toolbox (can be null)

**Returns:** Array of function call objects:
```javascript
[
  {
    name: 'domain.function_name',
    params: { param_name: value, ... },
    interface: <matching interface object>
  }
]
```

#### 3. `formatFunctionCallResults(functionCalls)`

Formats function execution results for display.

**Parameters:**
- `functionCalls`: Array of function call objects with results

**Returns:** Formatted string showing calls and results

### Text Format for Function Calls

When using a translator, functions are represented in this text format:

```
@domain.function_name( <*required_param>, <optional_param{type}> )
Description: What the function does
```

**Format Conventions:**
- `@` prefix indicates a function call
- `*` prefix on parameter means required
- `{type}` suffix shows non-string types (e.g., `{number}`, `{boolean}`)
- String types have no type suffix
- Optional `with_body [[ ]]` syntax for API wrappers with request bodies

**Examples:**
```
@weather.get_current_weather( <*location>, <unit> )
Description: Get current weather information for a location

@math.calculate_power( <*base{number}>, <*exponent{number}> )
Description: Calculate base raised to the power of exponent

@api.create_user( <*username>, <*email> ) with_body [[
{
  "username": "john_doe",
  "email": "john@example.com",
  "age": 30
}
]]
```

#### Output Filtering

The `atfunc` translator supports an optional output filter that can transform function results before they enter the AI context. This is useful for filtering noise from API responses.
```javascript
import { setOutputFilter } from '@semantictools/ai-toolbox/src/translators/atfunc.mjs';

setOutputFilter((functionName, params, result) => {
  // return modified result string, or null/undefined to keep original
});
```

**Example: Filtering feed spam**
```javascript
setOutputFilter((name, params, result) => {
  if (!result) return null;
  try {
    const parsed = JSON.parse(result);
    if (parsed.posts) {
      const original = parsed.posts.length;
      parsed.posts = parsed.posts.filter(p => !p.content?.includes('"op":"mint"'));
      if (parsed.posts.length < original) {
        parsed._filtered = `${original - parsed.posts.length} minting transactions hidden`;
      }
      return JSON.stringify(parsed);
    }
  } catch(e) {}
  return null;
});
```

### Setting Up a Translator

#### Using a Built-in Translator (Recommended)

```javascript
import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';

// Set translator BEFORE registering toolboxes (using constant)
setFunctionTranslator(TRANSLATOR_ATFUNC);

// Alternative: use translator name string
setFunctionTranslator('atfunc');

// Alternative: load explicitly
import { getTranslator, setFunctionTranslator } from '@semantictools/ai-toolbox';
const translator = getTranslator('atfunc');
setFunctionTranslator(translator);
```

#### Using a Custom Translator

```javascript
import * as translator from './my-translator.mjs';
import { setFunctionTranslator } from '@semantictools/ai-toolbox';

// Set translator BEFORE registering toolboxes
setFunctionTranslator(translator);
```

Your custom translator module must export:

```javascript
// Optional: Declare case sensitivity preference (default: true = case-sensitive)
export const caseSensitive = false;

export {
  generateFunctionIfText,
  getFunctionCallsFromText,
  formatFunctionCallResults
};
```

**Case Sensitivity Property:**
- `caseSensitive: true` - Exact case matching required (default)
- `caseSensitive: false` - Case-insensitive matching (like atfunc)
- If not specified, defaults to `true` (case-sensitive)

### Case Sensitivity

Case sensitivity is **determined by the translator** you use:

- **Default (no translator set)**: Case-sensitive
- **atfunc translator**: Case-insensitive (handles AI text variability)
- **Custom translators**: Specify via `caseSensitive` property

**How it works:**

```javascript
// Custom translator with case sensitivity preference
export const caseSensitive = false; // or true

export function generateFunctionIfText(...) { ... }
```

When you set a translator with `setFunctionTranslator()`, the toolmanager automatically adapts to its case sensitivity preference.

**Examples:**

```javascript
// Using atfunc (case-insensitive)
setFunctionTranslator(TRANSLATOR_ATFUNC);
register(myToolbox); // getName() returns 'Weather'

// All of these work:
call('weather.getTemp');
call('Weather.getTemp');
call('WEATHER.GETTEMP');
```

```javascript
// Custom case-sensitive translator
const myTranslator = {
  caseSensitive: true,
  generateFunctionIfText: ...,
  getFunctionCallsFromText: ...,
  formatFunctionCallResults: ...
};

setFunctionTranslator(myTranslator);
register(myToolbox); // getName() returns 'Weather'

// Only exact case works:
call('Weather.getTemp'); // ✓ Works
call('weather.getTemp'); // ✗ Error
```

**Important:** Set the translator **before** registering toolboxes for the case sensitivity to apply correctly.

### Why This Standard?

The interface standard was designed with these principles:

1. **Translator-Driven Case Handling**: Each translator declares its case sensitivity needs, toolmanager adapts
2. **AI Compatibility**: Text-based format enables natural integration with LLM function calling
3. **Consistency**: Standardized structure across all toolboxes improves maintainability
4. **Extensibility**: Supports both simple functions and complex API wrappers
5. **Type Safety**: Explicit type definitions help with parameter validation and conversion
6. **Production Tested**: Based on real-world implementations in production systems

### Reference Implementation

The `atfunc` translator is included as a built-in standard translator. You can use it directly with:

```javascript
import { setFunctionTranslator, TRANSLATOR_ATFUNC } from '@semantictools/ai-toolbox';

setFunctionTranslator(TRANSLATOR_ATFUNC);
```

This production-ready translator supports:
- Both `function` and `api-wrapper` types
- Type conversion (string, number, boolean, object)
- Quote and escape sequence handling
- Multi-line request bodies with `with_body [[ ]]` syntax
- Path parameters, query parameters, and request bodies

The complete interface specification is documented in this README. The reference implementation is available as a built-in translator accessible via `TRANSLATOR_ATFUNC`.

## Migration Guide

If you have existing code using the old interface format, here's how to migrate:

### Old Format (deprecated)
```javascript
interface: {
  name: 'getCurrentWeather',
  description: 'Get current weather',
  parameters: {
    location: {
      type: 'string',
      required: true,
      description: 'City name'
    }
  },
  returns: 'object'
}
```

### New Format (standardized)
```javascript
interface: {
  domain: 'weather',           // ← Add domain (must match toolbox name)
  name: 'get_current_weather', // ← Lowercase with underscores
  type: 'function',            // ← Add type
  description: 'Get current weather',
  params: {                    // ← Changed from 'parameters'
    location: {
      type: 'string',
      isRequired: true,        // ← Changed from 'required'
      description: 'City name'
    }
  },
  returns: 'object'
}
```

**Key Changes:**
1. Add `domain` field matching your toolbox name
2. Add `type` field (`'function'` or `'api-wrapper'`)
3. Convert function names to lowercase snake_case
4. Rename `parameters` to `params`
5. Rename `required` to `isRequired`

## Examples

See the [examples](./examples) directory for complete working examples using the standardized interface:

- [Basic Usage](./examples/basic.mjs) - Simple toolbox registration and function calls
- [Multiple Toolboxes](./examples/multiple-toolboxes.mjs) - Working with multiple toolboxes
- [AI Integration](./examples/ai-integration.mjs) - Full AI integration with translator
- [Simple Translator](./examples/simple-translator.mjs) - Basic translator implementation

**Built-in Translator:**
The `atfunc` translator is included as a built-in and can be used via the `TRANSLATOR_ATFUNC` constant or by name (`'atfunc'`). See the source at [src/translators/atfunc.mjs](./src/translators/atfunc.mjs).

## Standard Toolboxes

The package includes ready-to-use standard toolboxes that you can import and register:

### File Toolbox

A secure file operations toolbox that provides sandboxed access to a configured root folder.

**Features:**
- Operates within a configured root folder (prevents directory traversal)
- Two setup methods: `.filetb` file (beginners) or programmatic (production with custom settings)
- Support for basic file operations: read, write, delete, list, mkdir, rmdir

**Setup (Beginner - using .filetb file):**

```javascript
import { register, setRootToolBox } from '@semantictools/ai-toolbox';
import { fileToolbox, initialize } from '@semantictools/ai-toolbox/src/toolboxes/file.mjs';

// 1. Create a .filetb config file with your root folder path
// echo "/path/to/your/folder" > .filetb

// 2. Initialize the file toolbox
await initialize(); // Reads .filetb from current directory

// 3. Register the toolbox
register(fileToolbox);
setRootToolBox('file');
```

**Setup (custom - programmatic):**

```javascript
import { register, setRootToolBox } from '@semantictools/ai-toolbox';
import { fileToolbox, setRootFolder } from '@semantictools/ai-toolbox/src/toolboxes/file.mjs';

// No .filetb file needed!
const dataDir = process.env.DATA_DIR || '/var/app/data';
await setRootFolder(dataDir);

register(fileToolbox);
setRootToolBox('file');
```

**Available Functions:**

- `file.readfile(filepath)` - Read file contents
- `file.writefile(filepath, content)` - Write content to file
- `file.deletefile(filepath)` - Delete a file
- `file.listfiles(subdir?)` - List files and directories (optional subdirectory)
- `file.mkdir(dirpath)` - Create a directory
- `file.rmdir(dirpath)` - Remove an empty directory

**Example:**

```javascript
// Read a file
const content = await call('readfile', {
  filepath: { value: 'docs/readme.txt' }
});

// Write a file
await call('writefile', {
  filepath: { value: 'output/results.txt' },
  content: { value: 'Processing complete!' }
});

// List files
const files = await call('listfiles', {
  subdir: { value: 'data' }
});
```

See [examples/file-toolbox.mjs](./examples/file-toolbox.mjs) for a complete example.

## Use Cases

- **AI Function Calling**: Provide tools for Claude, GPT, or other AI models
- **Plugin Systems**: Create extensible plugin architectures
- **Service Orchestration**: Manage and call multiple service APIs
- **Workflow Automation**: Build flexible automation toolchains

## License

Apache-2.0

Copyright 2026 Dusty Wilhelm Murray / Semantic Tools

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Author

Dusty Wilhelm Murray / Semantic Tools