npm package: @orquesta/js-sdk
Description: Orquesta JS SDK for LLMOps and Remote Configurations
Author: orquestadev
Latest version: 1.3.3
Deprecated: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.

<p align="left">
  <a href="https://orquesta.cloud" target="_blank">
    <img src="https://raw.githubusercontent.com/orquestadev/orquesta-javascript/main/img/banner.png" alt="Orquesta"  height="84">
  </a>
</p>

_LLM Operations and Integration Platform_

![npm](https://img.shields.io/npm/v/%40orquesta%2Fjs-sdk)

# Orquesta Javascript SDK

## Contents

- [Installation](#installation)
- [Create a client instance](#createclient)
- [Usage](#usage)
- [Reference](#reference)

## Installation

<div id='installation'/>

```bash
npm install @orquesta/js-sdk
```

```bash
yarn add @orquesta/js-sdk
```

## Creating a client instance

<div id='createclient'/>

_You can get your workspace API key from the settings section in your Orquesta workspace. `https://my.orquesta.dev/<workspace>/settings/developers`_

Initialize the Orquesta client with your API key:

```ts
import { createClient } from '@orquesta/node';

const client = createClient({
  api_key: '__API_KEY__',
  ttl: 3600,
});
```

When creating a client instance, the following connection settings can be adjusted:

- `api_key`: string - your workspace API key to use for authentication.
- `environment?`: string - the environment to use for the client. Not required but recommended to use so it's added to the evaluation context automatically.
- `ttl?`: number - the time to live in seconds for the local cache. Default is 3600 seconds (1 hour).

## Usage - Endpoints

<div id='usage'/>

Use the Endpoints API to `query` or stream your endpoints from Orquesta.

Using endpoints to generate a LLM response based on your use case with Orquesta provides a low-latency, secure connection to the Endpoints API online prediction service. Getting out of the box metrics and logging for your LLMs.

Endpoints API support streaming and querying. The endpoint response type is `OrquestaEndpointResponse`. We recommend to use the code snippets provided in the Orquesta Admin panel to reduce risk of errors and improve ease of use.

#### Example: Querying an endpoint

```ts
const completion = await client.endpoints.query({
  key: 'customer_service',
  context: { environments: 'production', country: 'NLD' },
  variables: { firstname: 'John', city: 'New York' },
  metadata: { customer_id: 'Qwtqwty90281' },
});

console.log(completion.content);
```

#### Example: Streaming your endpoints

We provide support for streaming responses using Server Sent Events (SSE). The `stream` method output a `RxJS` observable that you can subscribe and receive the streaming chunks of data from the endpoint. The chunk type is `OrquestaEndpointResponse`

```ts
let endpointRef: OrquestaEndpoint;

const stream = client.endpoints
  .stream({
    key: 'customer_service',
    context: { environments: 'production', country: 'NLD' },
    variables: { firstname: 'John', city: 'New York' },
    metadata: { customer_id: 'Qwtqwty90281' },
  })
  .subscribe({
    next: (endpoint) => {
      endpointRef = endpoint;
      console.log(endpoint.content);
    },
    complete: () => {
      // Executed when the streaming is completed
    },
    error: (error) => {
      console.log(error);
    },
  });
```

If you need to cancel a stream, you can call `stream.unsubscribe()` method.

### Logging score and metadata for endpoints

After every query, Orquesta will generate a log with the result of the evaluation. You can add `metadata` and `score` to the endpoint by using the `addMetrics` method.

```ts
endpointRef.addMetrics({
  score: 85,
  metadata: {
    custom: 'custom_metadata',
    chain_id: 'ad1231xsdaABw',
  },
});
```

## Usage - Prompts

<div id='usage'/>

Use the Prompts API to query your prompts from Orquesta.

You can use Orquesta in prompt management mode by consuming our Prompts API. The prompt value type is `OrquestaPrompt`. We recommend to use the code snippets provided in the Orquesta Admin panel to reduce risk of errors and improve ease of use.

We support an unified data model structure for all our prompts and provide helper functions that map the returned value from Orquesta to the specific provider.

The `query` method receives an object of type `OrquestaPromptRequest` as parameter.

#### Example: Querying a prompt

```ts
import { orquestaOpenAIMapper } from '@orquesta/utils';
import { OrquestaOpenAIPromptParameters, OrquestaPrompt } from '@orquesta/core';

const prompt: OrquestaPrompt = await client.prompts.query({
  key: 'prompt_key',
  context: { environments: 'production', country: 'NLD' },
  variables: { firstname: 'John', city: 'New York' },
  metadata: { chain_id: 'ad1231xsdaABw' },
});

const openAIPrompt: OrquestaOpenAIPromptParameters =
  orquestaOpenAIParametersMapper(prompt.value);
```

#### Helper functions per LLM provider

We provide `helper` functions and `interfaces` that map the returned value from Orquesta to the specific provider, so it's easy for you to forward the Prompt to your different LLM providers.

| Provider     | Helper                              | Interface                           |
| ------------ | ----------------------------------- | ----------------------------------- |
| Anthropic    | `orquestaAnthropicParametersMapper` | `OrquestaAnthropicPromptParameters` |
| Cohere       | `orquestaCohereParametersMapper`    | `OrquestaCoherePromptParameters`    |
| Google       | `orquestaGoogleParametersMapper`    | `OrquestaGooglePromptParameters`    |
| OpenAI       | `orquestaOpenAIParametersMapper`    | `OrquestaOpenAIPromptParameters`    |
| Hugging Face | `⚠️ Work in progres`                |                                     |
| Replicate    | `⚠️ Work in progres`                |                                     |

### Logging responses and metadata for prompts

After every query, Orquesta will generate a log with the result of the evaluation. You can add metadata and information about the interaction with the LLM to the log by using the `addMetrics` method.

The properties `score`, `latency`, `llm_response` and `economics` are reserved and used to generate your real-time dashboards. `metadata` is a set of key-value pairs that you can use to add custom information to the log.

#### Example: Add metrics to your request log

```ts
prompt.addMetrics({
  score: 85,
  latency: 4000,
  llm_response: 'Orquesta is awesome!',
  economics: {
    prompt_tokens: 1200,
    completion_tokens: 750,
    total_tokens: 1950,
  },
  metadata: {
    custom: 'custom_metadata',
    chain_id: 'ad1231xsdaABw',
    total_interactions: 200,
  },
});
```

## Usage - Remote Configurations

Orquesta has a powerful Remote Configurations API that allows you to configure and run all your environments and services remotely dynamically. Orquesta supports different types of remote configurations, and we recommend always typing the `query` method to help Typescript infer the correct type.

The `useOrquestaRemoteConfig` hook receives an object of type `OrquestaRemoteConfigQuery` as parameter.

Supported types: `boolean`, `number`, `string`, `json`, `array`

#### Example: Querying a configuration of type boolean

```ts
const config: OrquestaRemoteConfig<boolean> =
  await client.remoteConfigs.query<boolean>({
    key: 'boolean_config',
    default_value: false,
    context: { environments: 'production', role: 'admin' },
    metadata: { timestamp: Date.now() },
  });
```

#### Example: Querying a configuration of type string

```ts
const config: OrquestaRemoteConfig<string> =
  await client.remoteConfigs.query<string>({
    key: 'string_config',
    default_value: 'string_value',
    context: { environments: 'production', country: 'NL' },
    metadata: { timestamp: Date.now() },
  });
```

#### Example: Querying a configuration of type number

```ts
const config: OrquestaRemoteConfig<number> =
  await client.remoteConfigs.query<number>({
    key: 'number_config',
    default_value: 1990,
    context: { environments: 'production', market: 'US', domain: 'ecommerce' },
    metadata: { timestamp: Date.now() },
  });
```

#### Example: Querying a configuration of type array

```ts
const config: OrquestaRemoteConfig<string[]> = await client.remoteConfigs.query<
  string[]
>({
  key: 'array_config',
  default_value: ['value1', 'value2'],
  context: { environments: 'acceptance', isEnable: true },
  metadata: { timestamp: Date.now() },
});
```

#### Example: Querying a configuration of type json

It's recommended to add an interface to the json configuration to help Typescript infer the correct type.

```ts
const config: OrquestaRemoteConfig<object> =
  await client.remoteConfigs.query<object>({
    key: 'json_config',
    default_value: { dashboardEnabled: false, theme: 'dark' },
    context: { environments: 'develop', platform: 'mobile' },
    metadata: { timestamp: Date.now() },
  });
```

### Additional metadata logging

After every query, Orquesta will generate a log with data about the request. You can add metadata to the log using the `addMetrics` method anytime.

`metadata` is a set of `key-value pairs` that you can use to add custom information to the log.

#### Example: Add metrics to your request log

```ts
prompt.addMetrics({
  metadata: {
    custom: 'custom_metadata',
    user_clicks: 20,
    selected_option: 'option1',
  },
});
```

<div id='references'/>

# Orquesta API

## Endpoints API

Types:

- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/endpoints.ts#L6">OrquestaEndpoint</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/endpoints.ts#L127">OrquestaEndpointMetrics</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/endpoints.ts#L120">OrquestaEndpointRequest</a></code>

Methods:

- <code>client.endpoints.<a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/endpoints.ts#L137">query</a>({ ...params }) -> Promise`<OrquestaEndpoint>`</code>
- <code>client.endpoints.<a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/endpoints.ts#L159">stream</a>({ ...params }) -> Observable`<OrquestaEndpoint>` </code>

## Prompts API

Types:

- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/prompts.ts#L31">OrquestaPrompt</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/prompts.ts#L23">OrquestaPromptMetrics</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/prompts.ts#L17">OrquestaPromptMetricsEconomics</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/prompts.ts#L10">OrquestaPromptRequest</a></code>

Methods:

- <code>client.prompts.<a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/prompts.ts#L53">query</a>({ ...params }) -> Promise`<OrquestaPrompt>`</code>

## RemoteConfigs API

Types:

- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/remoteconfigs.ts#L20">OrquestaRemoteConfig</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/remoteconfigs.ts#L9">OrquestaRemoteConfigMetrics</a></code>
- <code><a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/remoteconfigs.ts#L13">OrquestaRemoteConfigRequest</a></code>

Methods:

- <code>client.remoteconfigs.<a href="https://github.com/orquestadev/orquesta-javascript/blob/main/packages/js/src/lib/remoteconfigs.ts#L38">query<T></a>({ ...params }) -> Promise`<OrquestaRemoteConfig<T>>`</code>