Repository: promptfoo/promptfoo
Issue: Request: supporting embeddings endpoint for LocalAI and Ollama
Root cause: Unimplemented Feature Gap

LocalAI has an embeddings endpoint ([source](https://github.com/go-skynet/LocalAI/blob/7bdf707dd3d4eaeb09eea31484ab2f51fba1ba32/api/api.go#L195), [docs](https://localai.io/features/embeddings/)):

```bash
> curl http://localhost:8080/embeddings -X POST -H "Content-Type: application/json" -d '{
    "input": "Your text string goes here",
    "model": "bert-embeddings"
  }' | jq "."
...
{
  "object": "list",
  "model": "bert-embeddings",
  "data": [
    {
      "embedding": [
        0.051830754,
        ...,
        -0.023937061
      ],
      "index": 0,
      "object": "embedding"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}
```

Ollama also has one: [`/api/embeddings`](https://github.com/jmorganca/ollama/blob/v0.0.19/server/routes.go#L527)

Trying to use a LocalAI model for grader:

```yaml
defaultTest:
    options:
        provider: localai:chat:bert-embeddings
```

I get this error:

```none
[FAIL] Error: Invariant failed: Provider must implement callEmbeddingApi for similarity check
Error: Invariant failed: Provider must implement callEmbeddingApi for similarity check
at invariant (/Users/user/code/promptfoo/node_modules/tiny-invariant/dist/tiny-invariant.cjs.js:14:11)
at matchesSimilarity (/Users/user/code/promptfoo/dist/src/matchers.js:63:34)
at async runAssertion (/Users/user/code/promptfoo/dist/src/assertions.js:367:17)
at async runAssertions (/Users/user/code/promptfoo/dist/src/assertions.js:49:24)
at async Evaluator.runEval (/Users/user/code/promptfoo/dist/src/evaluator.js:182:37)
at async /Users/user/code/promptfoo/dist/src/evaluator.js:418:25
```

The only providers right now (0.24.1) that implemented `callEmbeddingApi` are OpenAI and Azure.

Can we add `callEmbeddingApi` to the LocalAI and Ollama providers?  Thank you in advance

@jamesbraza I snuck this change into the latest patch release (0.24.2).  I admit I did not test it fully, as I'm traveling and my current setup won't support local models, so I'm not going to close this issue yet.  But #189 adds support for `ollama:embedding:...` and `localai:embedding:...`

Yeah I just saw that, very much appreciated!!  Impressed you're so productive while travelling. Feel free to tag me on PRs if wanted ever btw.

> adds support for `ollama:embedding:...` and `localai:embedding:...`

Just for people reading this in posterity, it's technically `ollama:embeddings:...` and `localai:embeddings:...`.

I manually tested LocalAI embeddings now and it works! 🥳 Going to close this out, if I encounter issues I will open a separate ticket

> Yeah I just saw that, very much appreciated!! Impressed you're so productive while travelling. Feel free to tag me on PRs if wanted ever btw.
> 
> > adds support for `ollama:embedding:...` and `localai:embedding:...`
> 
> Just for people reading this in posterity, it's technically `ollama:embeddings:...` and `localai:embeddings:...`.
> 
> I manually tested LocalAI embeddings now and it works! 🥳 Going to close this out, if I encounter issues I will open a separate ticket

Hi @jamesbraza  i'm try use provider: [ollama:embeddings:llama3:8b] still face error 
Error: Provider must be an object or string, but received an array

is it same on your local ? thanks

I think you may be accidentally writing `- provider:` instead of `provider:`