Repository: promptfoo/promptfoo
Issue: SqliteError: FOREIGN KEY constraint failed (starting in 0.92)
Root cause: External Dependency Breakage

**Describe the bug**

Upgrading from 0.91.0 to 0.92.0 (or 0.93.3) causes a foreign key error when using the node package with "writeLatestResults: true".

I'm using the `promptfoo` node package via a small node script (run using `tsx`). It's broken since upgrading to 0.92 or above. If I remove "writeLatestResults: true" the eval script finishes (but then isn't viewable).

**To Reproduce**
Steps to reproduce the behavior, including example Promptfoo configurations if applicable:

1. Install `promptfoo` 0.92 or higher and `tsx`
2. Run the eval script (pasted below): `tsx ./run-eval.ts my-test-suite.ts`
3. See error below

**Expected behavior**
Running an eval completes without errors and saves results for the web viewer. Works as expected with version <= 0.91.0.

**Screenshots**
If applicable, add screenshots to help explain your problem.

run-eval.ts:
```
import path from "path";
import { evaluate, EvaluateOptions, EvaluateTestSuite } from "promptfoo";

const options: EvaluateOptions = {
  maxConcurrency: 5,
};

/**
 * Run an eval and print the results.
 *
 * The promptfoo CLI didn't work well with Typescript / ES modules, so we wrap the node library instead.
 *
 * Usage:
 * npm run eval <path to eval file>
 */
async function runEval() {
  const evalFilePath = process.argv[2];
  if (!evalFilePath) {
    console.error("Please provide the path to an eval file");
    process.exit(1);
  }

  const { testSuite } = await import(path.resolve(evalFilePath));

  const finalTestSuite: EvaluateTestSuite = {
    // Write the latest results to promptfoo storage to use the web viewer.
    writeLatestResults: true,

    ...testSuite,
  };

  const evalResults = await evaluate(finalTestSuite, options);
  const summary = await evalResults.toEvaluateSummary();
  console.log(summary.stats);
}

runEval();
```

Error logged to console:
```
Error saving result: SqliteError: FOREIGN KEY constraint failed {"provider":{"id":"claude-3-5-sonnet-20240620"},"prompt":{"raw":"[\n  {\n    \"role\": \"system\",\n    \"content\": \"<...SYSTEM...>\",\n    \"experimental_providerMetadata\": {\n      \"anthropic\": {\n        \"cacheControl\": {\n          \"type\": \"ephemeral\"\n        }\n      }\n    }\n  },\n  {\n    \"role\": \"user\",\n    \"content\": \"foo bar\"\n  }\n]","label":""},"vars":{"promptText":"foo bar"},"response":{"output":"Sorry, that prompt doesn't provide enough context to generate LaTeX code. Please try again.","cost":0.000477,"tokenUsage":{"total":39,"prompt":9,"completion":30}},"success":true,"score":1,"namedScores":{"Returns an error":1},"latencyMs":2747,"cost":0.000477,"promptIdx":0,"testIdx":0,"testCase":{"vars":{"promptText":"foo bar"},"options":{"transform":"output.error"},"assert":[{"metric":"Returns an error","type":"javascript"}],"metadata":{}},"promptId":"","gradingResult":{"pass":true,"score":1,"reason":"All assertions passed","namedScores":{"Returns an error":1},"tokensUsed":{"total":0,"prompt":0,"completion":0,"cached":0},"componentResults":[{"pass":true,"score":1,"reason":"Returns an error","assertion":{"type":"javascript","value":"output=>buildGradingResult(assertion(output),metri..."}}],"assertion":null}}
```

**System information:**

- Promptfoo version: 0.92.0 or 0.93.3 don't work (0.91.0 does)

**Additional context**

Looks like a similar issue to #1862

(thanks for the great library!)

@holdenmatt I've got a fix for you here: https://github.com/promptfoo/promptfoo/pull/1955

Great, thanks for the quick fix!

Do you know what version this will be in?

When it's live on npm, I'll upgrade and test it again.

@holdenmatt we’ll cut a new release within the next ~18 hours. Once it’s live, I’ll update it here so you can upgrade and test. Thanks for your patience!

Hi @holdenmatt, the fix for this issue is now available in version [0.94.0](https://github.com/promptfoo/promptfoo/releases/tag/0.94.0). Please upgrade and let us know if it resolves the problem.

Thanks again for your patience and for using Promptfoo!

Confirmed that error is fixed, thanks!

Getting a type error now, but guessing something just changed in the types. I'll work on tracking that down, but can call this closed.

```
Warning: Unsupported 'tests' format in promptfooconfig.yaml.
Expected: string, string[], or TestCase[], but received: undefined

Please check your configuration file and ensure the 'tests' field is correctly formatted.
For more information, visit: https://promptfoo.dev/docs/configuration/reference/#test-case
```

@holdenmatt thanks for reporting! I added the warning in the last release. I've fixed this in https://github.com/promptfoo/promptfoo/commit/c1a86d599353f582ccdc882aa13aa29068c014f3 - the warning will no longer show when tests are undefined. This fix will go out in the next release.