Repository: promptfoo/promptfoo
Issue: Evals Page Empty v0.92.3
Root cause: Interface Contract Mismatch

**Describe the bug**
The `Evals` tab on the self-hosted UI is always empty, no matter how many evals I run.

**To Reproduce**
Steps to reproduce the behavior, including example Promptfoo configurations if applicable:

1. Deployed self-hosted docker container 0.92.3.
2. Ran example evaluations from information outlined here: https://www.promptfoo.dev/docs/getting-started/

**Expected behavior**
The `Evals` tab shows all the recent evaluations.

**Screenshots**
![image](https://github.com/user-attachments/assets/114a84d0-bc99-4a3d-9d28-c693b8d7f2c9)
![image](https://github.com/user-attachments/assets/556a0904-e080-4068-8d77-489f10c14ab8)

Interestingly enough, I can see a data object returned in a network request, and it contains a list of the evaluations I ran.

- Promptfoo version: 0.92.3



Hi @bwhartlove, thanks for reporting this issue! To help us investigate, could you please confirm a couple of things:

1. Did you map the volume correctly when running the Docker container, specifically using the `-v /path/to/local_promptfoo:/root/.promptfoo` flag? This is where the SQLite database lives and is needed for persisting evals.
2. Did you override any other environment variables when starting the container?

We'll look into this right away. Thanks! 

Are there any errors or messages in the console tab?

@mldangelo  Thanks for the quick reply!

1. I created an empty directory at `/data/packages` and mounted that to `/root/.promptfoo` when spinning up the container. I can see the sqlite db there.
2. The only ENV vars I overrode was the `PROMPTFOO_REMOTE_API_BASE_URL` and `PROMPTFOO_REMOTE_APP_BASE_URL` to match my IP:Port scheme.
@sklein12  I don't see anything in the console besides what appears to be an info message:
`Eval init: Fetching eval via recent`

@bwhartlove can you try clearing the local key value store? Just click `delete database` on this screen.

<img width="824" alt="image" src="https://github.com/user-attachments/assets/f18586aa-f64c-4695-8595-c03ac8355933">


@sklein12 Tried that, but no cigar. Still the same message in the console and new evals on my page. 

To note, I can 'get' to the eval by going to the datasets tab and clicking on the most recent eval. It just seems like the eval page itself is not showing the evaluations.

@bwhartlove ok thanks! Can you share with me the raw json from that network request you mentioned in your original post? That'll be the easiest way for me to repro.

@sklein12  Sure! Here is the example JSON that came back from the network request previously mentioned:
![image](https://github.com/user-attachments/assets/e5e7d70e-1966-43a7-96dc-fa60503038fe)


> @sklein12 Sure! Here is the example JSON that came back from the network request previously mentioned: ![image](https://private-user-images.githubusercontent.com/110192746/375982374-e5e7d70e-1966-43a7-96dc-fa60503038fe.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mjg3NjA3NzMsIm5iZiI6MTcyODc2MDQ3MywicGF0aCI6Ii8xMTAxOTI3NDYvMzc1OTgyMzc0LWU1ZTdkNzBlLTE5NjYtNDNhNy05NmRjLWZhNjA1MDMwMzhmZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQxMDEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MTAxMlQxOTE0MzNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NjZhYTQ3NGU2YzM3YTAyZjk0MzBhNThjZDZmNGVmMDE0ZjAzM2IxNjk4Zjg3MDgwODQ2NzI1MWJjNjYwYTE1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.y8LUuxb2GyQNY5zMtNwiAfu_rB78xCjlOQfaQYEM7M8)

@bwhartlove can you copy the entire response as text for me?

Yeah here is the entire raw JSON back from the response:
```
{"data":[{"evalId":"eval-n2h-2024-10-12T19:13:24","createdAt":1728760404343,"description":"Example promptfoo evaluation","numTests":2,"datasetId":"f76f0be5e2f2d66234ec747945bc9984c590415a0c9bf9b15fbf60e582d6f57d","label":"Example promptfoo evaluation (eval-n2h-2024-10-12T19:13:24)"}]}
```


> Yeah here is the entire raw JSON back from the response:
> 
> ```
> {"data":[{"evalId":"eval-n2h-2024-10-12T19:13:24","createdAt":1728760404343,"description":"Example promptfoo evaluation","numTests":2,"datasetId":"f76f0be5e2f2d66234ec747945bc9984c590415a0c9bf9b15fbf60e582d6f57d","label":"Example promptfoo evaluation (eval-n2h-2024-10-12T19:13:24)"}]}
> ```

@bwhartlove is there another request being made after the results request?

Yes, in fact there is. I see one more request after the results request to the endpoint `/api/results/eval-HAL-2024-10-12T19:51:10`.  Here's the raw response: 
```
{"data":{"version":4,"createdAt":"2024-10-12T19:51:10.622Z","results":{"version":3,"timestamp":"2024-10-12T19:51:10.622Z","prompts":[{"raw":"Convert this English to {{language}}: {{input}}","label":"Convert this English to {{language}}: {{input}}","id":"982e2daba2e45daa63007e0713738f8cb47c07a372becd90661ec209f021cc76","provider":"openai:chat:/inference_model","metrics":{"score":2,"testPassCount":2,"testFailCount":0,"assertPassCount":0,"assertFailCount":0,"totalLatencyMs":2908,"tokenUsage":{"total":94,"prompt":39,"completion":55,"cached":0},"namedScores":{},"namedScoresCount":{},"cost":0}},{"raw":"Translate to {{language}}: {{input}}","label":"Translate to {{language}}: {{input}}","id":"1cd6d24599bd30b06147480d8c35ca4b3ffb0c41385df8f0624099444fb68ae8","provider":"openai:chat:/inference_model","metrics":{"score":2,"testPassCount":2,"testFailCount":0,"assertPassCount":0,"assertFailCount":0,"totalLatencyMs":904,"tokenUsage":{"total":49,"prompt":35,"completion":14,"cached":0},"namedScores":{},"namedScoresCount":{},"cost":0}}],"results":[{"cost":0,"gradingResult":{"pass":true,"score":1,"reason":"No assertions","tokensUsed":{"total":0,"prompt":0,"completion":0,"cached":0},"assertion":null},"id":"511d64f0-14ba-465a-a01c-22df65379fb8","latencyMs":305,"namedScores":{},"prompt":{"raw":"Convert this English to French: Hello world","label":"Convert this English to {{language}}: {{input}}"},"promptId":"982e2daba2e45daa63007e0713738f8cb47c07a372becd90661ec209f021cc76","promptIdx":0,"provider":{"id":"openai:chat:/inference_model","label":""},"response":{"output":{"role":"assistant","content":"Bonjour le monde","tool_calls":[]},"tokenUsage":{"total":22,"prompt":18,"completion":4},"cached":false},"score":1,"success":true,"testCase":{"vars":{"language":"French","input":"Hello world"},"assert":[],"options":{},"metadata":{}},"testIdx":0,"vars":{"language":"French","input":"Hello world"}},{"cost":0,"gradingResult":{"pass":true,"score":1,"reason":"No assertions","tokensUsed":{"total":0,"prompt":0,"completion":0,"cached":0},"assertion":null},"id":"59af8117-2af9-4ae2-9823-730c1054d7ea","latencyMs":308,"namedScores":{},"prompt":{"raw":"Translate to French: Hello world","label":"Translate to {{language}}: {{input}}"},"promptId":"1cd6d24599bd30b06147480d8c35ca4b3ffb0c41385df8f0624099444fb68ae8","promptIdx":1,"provider":{"id":"openai:chat:/inference_model","label":""},"response":{"output":{"role":"assistant","content":"Bonjour le monde","tool_calls":[]},"tokenUsage":{"total":20,"prompt":16,"completion":4},"cached":false},"score":1,"success":true,"testCase":{"vars":{"language":"French","input":"Hello world"},"assert":[],"options":{},"metadata":{}},"testIdx":0,"vars":{"language":"French","input":"Hello world"}},{"cost":0,"gradingResult":{"pass":true,"score":1,"reason":"No assertions","tokensUsed":{"total":0,"prompt":0,"completion":0,"cached":0},"assertion":null},"id":"63270f89-a786-44b4-8d18-32ef3436ade2","latencyMs":596,"namedScores":{},"prompt":{"raw":"Translate to Spanish: Where is the library?","label":"Translate to {{language}}: {{input}}"},"promptId":"1cd6d24599bd30b06147480d8c35ca4b3ffb0c41385df8f0624099444fb68ae8","promptIdx":1,"provider":{"id":"openai:chat:/inference_model","label":""},"response":{"output":{"role":"assistant","content":"¿Dónde está la biblioteca?","tool_calls":[]},"tokenUsage":{"total":29,"prompt":19,"completion":10},"cached":false},"score":1,"success":true,"testCase":{"vars":{"language":"Spanish","input":"Where is the library?"},"assert":[],"options":{},"metadata":{}},"testIdx":1,"vars":{"language":"Spanish","input":"Where is the library?"}},{"cost":0,"gradingResult":{"pass":true,"score":1,"reason":"No assertions","tokensUsed":{"total":0,"prompt":0,"completion":0,"cached":0},"assertion":null},"id":"3119ec7f-1e83-4fc8-a5f3-4dcefc410a0c","latencyMs":2603,"namedScores":{},"prompt":{"raw":"Convert this English to Spanish: Where is the library?","label":"Convert this English to {{language}}: {{input}}"},"promptId":"982e2daba2e45daa63007e0713738f8cb47c07a372becd90661ec209f021cc76","promptIdx":0,"provider":{"id":"openai:chat:/inference_model","label":""},"response":{"output":{"role":"assistant","content":"The translation of \"Where is the library?\" in Spanish is:\n\n¿Dónde está la biblioteca?\n\n(Note: The upside-down question mark at the beginning of the sentence is a characteristic of Spanish punctuation, indicating that the sentence is a question.)","tool_calls":[]},"tokenUsage":{"total":72,"prompt":21,"completion":51},"cached":false},"score":1,"success":true,"testCase":{"vars":{"language":"Spanish","input":"Where is the library?"},"assert":[],"options":{},"metadata":{}},"testIdx":1,"vars":{"language":"Spanish","input":"Where is the library?"}}],"stats":{"successes":4,"failures":0,"tokenUsage":{"cached":0,"completion":69,"prompt":74,"total":143}}},"config":{"defaultTest":{},"description":"Example promptfoo evaluation","env":{},"prompts":[{"raw":"Convert this English to {{language}}: {{input}}","label":"Convert this English to {{language}}: {{input}}"},{"raw":"Translate to {{language}}: {{input}}","label":"Translate to {{language}}: {{input}}"}],"providers":[{"id":"openai:chat:/inference_model","config":{"temperature":0.01,"max_tokens":2048,"apiBaseUrl":"http://192.168.1.55:3000/v1","apiKey":"-"}}],"scenarios":[],"tests":[{"vars":{"language":"French","input":"Hello world"}},{"vars":{"language":"Spanish","input":"Where is the library?"}}],"writeLatestResults":true,"sharing":true},"author":null,"prompts":[{"raw":"Convert this English to {{language}}: {{input}}","label":"Convert this English to {{language}}: {{input}}","id":"982e2daba2e45daa63007e0713738f8cb47c07a372becd90661ec209f021cc76","provider":"openai:chat:/inference_model","metrics":{"score":2,"testPassCount":2,"testFailCount":0,"assertPassCount":0,"assertFailCount":0,"totalLatencyMs":2908,"tokenUsage":{"total":94,"prompt":39,"completion":55,"cached":0},"namedScores":{},"namedScoresCount":{},"cost":0}},{"raw":"Translate to {{language}}: {{input}}","label":"Translate to {{language}}: {{input}}","id":"1cd6d24599bd30b06147480d8c35ca4b3ffb0c41385df8f0624099444fb68ae8","provider":"openai:chat:/inference_model","metrics":{"score":2,"testPassCount":2,"testFailCount":0,"assertPassCount":0,"assertFailCount":0,"totalLatencyMs":904,"tokenUsage":{"total":49,"prompt":35,"completion":14,"cached":0},"namedScores":{},"namedScoresCount":{},"cost":0}}],"datasetId":"f76f0be5e2f2d66234ec747945bc9984c590415a0c9bf9b15fbf60e582d6f57d"}}
```

@bwhartlove can you book 30 minutes on my calendar next week? We can debug it via zoom https://cal.com/steve-klein-10xzxs/30min

@sklein12  Let me check in with my team next week and get a time that would work. Thank you!

As a somewhat separate and hopefully quick issue - If I run promptfoo self-hosted via Docker, is there a way to use the node package on the CLI to store it's eval results in the self-hosted application? From what I can tell, when I use `promptfoo eval` it stores the information in a cache directory and uses those results with `promptfoo view`. I'd like a programatic way to run evaluations that doesn't include using the UI to kick them off, but still have a central location to view and compare results. Is that currently feasible?

@bwhartlove definitely, you can set the sharing property inside the TestSuiteConfig

https://www.promptfoo.dev/docs/configuration/reference/#testsuiteconfiguration

```
 // Determines whether or not sharing is enabled.
  sharing?:
    | boolean
    | {
        apiBaseUrl?: string;
        appBaseUrl?: string;
      };
  ```

Would this be a correct way to set that up? I didn't see the results reflected in my app using this, but the eval ran successfully.

```
const results = await promptfoo.evaluate(
  {
    prompts: ['Rephrase this in French: {{body}}', 'Rephrase this like a pirate: {{body}}'],
    providers: providers,
    tests: tests,
    sharing: {
        apiBaseUrl: "http://192.168.1.55:3000",
        appBaseUrl: "http://192.168.1.55:3000"
    }
  },
);
```

> Would this be a correct way to set that up? I didn't see the results reflected in my app using this, but the eval ran successfully.
> 
> ```
> const results = await promptfoo.evaluate(
>   {
>     prompts: ['Rephrase this in French: {{body}}', 'Rephrase this like a pirate: {{body}}'],
>     providers: providers,
>     tests: tests,
>     sharing: {
>         apiBaseUrl: "http://192.168.1.55:3000",
>         appBaseUrl: "http://192.168.1.55:3000"
>     }
>   },
> );
> ```

that should do it

Fwiw, I merged a fix for this issue.

@sklein12  Thanks for the fix! I was able to run an eval from the UI and my evals now show up in the eval tab as exepected.

That sharing thing still does not seem to work, though. I don't see anything show up in the UI when setting those sharing Urls.

@bwhartlove you're right, sharing isn't implemented for the node package. That's something we'll need to implement. I'll check with the team to see when we can get to it.

We'd also love your contribution to the project if you'd like to implement it.