Repository: promptfoo/promptfoo
Issue: CustomProvider (javascript) not caching results
Root cause: Documentation Deficiency

Hi there!

I've implemented a custom provider, it runs well, but unfortunately the results don't seem to be caching. It may be related to this: https://github.com/promptfoo/promptfoo/issues/730

Here's the code for reference:
```javascript
// customApiProvider.mjs
import { generateObject } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { promptSchema } from "./schemaValidation.mjs";

class CustomProvider {
  constructor(options) {
    // Provider ID can be overridden by the config file (e.g. when using multiple of the same provider)
    this.providerId = options.id || "custom provider";

    // options.config contains any custom options passed to the provider
    this.config = options.config;
  }

  id() {
    return this.providerId;
  }

  async callApi(prompt, context) {
    const model = anthropic("claude-3-haiku-20240307");
    const { object, usage } = await generateObject({
      model,
      messages: JSON.parse(prompt),
      maxTokens: 4096,
      maxRetries: 0,
      schema: promptSchema,
      mode: "tool"
    });

    const inputCost = 0.00025 / 1000;
    const outputCost = 0.00125 / 1000;

    return {
      // Required
      output: object,

      // Optional
      tokenUsage: {
        total: usage.totalTokens,
        prompt: usage.promptTokens,
        completion: usage.completionTokens
      },

      cost: inputCost * usage.promptTokens + outputCost * usage.completionTokens || undefined
    };
  }
}

export default CustomProvider;
```
and
```yaml
providers: ["file://customProvider.mjs"]
```

Or is there something missing in the config there?

Thanks for all your awesome work. 🙌

Thanks for the feedback!  #923 exposes the `cache` module on the promptfoo object, so you can use `promptfoo.cache.getCache()` get/set or `promptfoo.cache.fetchWithCache` to interact with the cache.

> Thanks for the feedback! #923 exposes the `cache` module on the promptfoo object, so you can use `promptfoo.cache.getCache()` get/set or `promptfoo.cache.fetchWithCache` to interact with the cache.

Thanks for the quick turnaround! I got it working, but there seems to be a small typing issue. With ES modules I can do this: `import promptfoo from "promptfoo"`, and then thought I could just use: `const cache = await promptfoo.cache.getCache();` (as the typings indicate). However, it needs to be `const cache = await promptfoo.default.cache.getCache();` to correctly work. 

Here's the full file for anyone else landing here needing to implement the "ai" lib:
```javascript
// @ts-check
import { generateObject } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { promptSchema } from "./schemaValidation.mjs";
import promptfoo from "promptfoo";

class CustomProvider {
  constructor(options) {
    // Provider ID can be overridden by the config file (e.g. when using multiple of the same provider)
    this.providerId = options.id || "custom provider";

    // options.config contains any custom options passed to the provider
    this.config = options.config;
  }

  id() {
    return this.providerId;
  }

  async callApi(prompt, context) {
    const cache = await promptfoo.default.cache.getCache();

    // Create a unique cache key based on the prompt and context
    const cacheKey = `api:${this.providerId}:${prompt}}`; // :${JSON.stringify(context)

    // Check if the response is already cached
    const cachedResponse = await cache.get(cacheKey);
    if (cachedResponse) {
      return {
        // Required
        output: JSON.parse(cachedResponse),

        // Optional
        tokenUsage: {
          total: 0, // No tokens used because it's from the cache
          prompt: 0,
          completion: 0
        },

        cost: 0 // No cost because it's from the cache
      };
    }

    // If not cached, make the function call
    const model = anthropic("claude-3-haiku-20240307");
    const { object, usage } = await generateObject({
      model,
      messages: JSON.parse(prompt),
      maxTokens: 4096,
      temperature: 0.4,
      maxRetries: 0,
      schema: promptSchema,
      mode: "tool"
    });

    const inputCost = 0.00025 / 1000; //config.cost ?? model.cost.input;
    const outputCost = 0.00125 / 1000; // config.cost ?? model.cost.output;
    const totalCost =
      inputCost * usage.promptTokens + outputCost * usage.completionTokens || undefined;

    // Store the response in the cache
    try {
      await cache.set(cacheKey, JSON.stringify(object));
    } catch (error) {
      console.error("Failed to store response in cache:", error);
    }

    return {
      // Required
      output: object,

      // Optional
      tokenUsage: {
        total: usage.totalTokens,
        prompt: usage.promptTokens,
        completion: usage.completionTokens
      },

      cost: totalCost
    };
  }
}

export default CustomProvider;
```

Thanks again. 🙌

Thanks @Codeshark-NET!  I added this to the `custom-provider` example directory.