Repository: promptfoo/promptfoo
Issue: Getting ECONNREFUSED ::1:11434 with Ollama provider
Root cause: Documentation Deficiency

I have the Ollama server running on port 11434 on my macOS (M1, Ventura 13.5.2), and I can `curl` it per their [docs](https://github.com/jmorganca/ollama#rest-api):

```bash
> curl -X POST -H "Content-Type: application/json" http://localhost:11434/api/generate -d '{
  "model": "llama2:13b",
  "prompt":"Why is the sky blue?"
}'
{"model":"llama2:13b","created_at":"2023-09-20T19:15:28.501932Z","response":" The","done":false}
{"model":"llama2:13b","created_at":"2023-09-20T19:15:28.546594Z","response":" sky","done":false}
{"model":"llama2:13b","created_at":"2023-09-20T19:15:28.599186Z","response":" appears","done":false}
{"model":"llama2:13b","created_at":"2023-09-20T19:15:28.651228Z","response":" blue","done":false}
{"model":"llama2:13b","created_at":"2023-09-20T19:15:28.702868Z","response":" because","done":false}
```

Note the responses don't start coming in for about 15 seconds.

However, when running `npm run promptfoo eval` (`promptfoo` 0.24.0)  I get:

```
> promptfoo eval --verbose
Eval: [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 0% | ETA: 0s | 0/2 |  "" Calling Ollama API: {"model":"llama2:13b","prompt":"Rephrase this from English to Pirate: Yes"}
Calling Ollama API: {"model":"llama2:13b","prompt":"Pretend you're a pirate and speak these words: Yes"}
...
```

The Ollama server never shows any activity in logs, and then viewing output with `promptfoo view`:

```
[FAIL] API call error: Error: Request failed after 4 retries: request to http://localhost:11434/api/generate failed, reason: connect ECONNREFUSED ::1:11434
```

What might be the problem here?  My `promptfooconfig.yaml` is:

```yaml
---
prompts: [prompts.txt]
providers: [ollama:llama2:13b]
tests:
    - description: Study areas
      vars:
          tech: A
          area: B
```

And my `prompts.txt`:

```txt
I am working with {{ tech }} to study {{ area }}. What is the area?
```

Damn, figured it out, inspired by [this](https://stackoverflow.com/a/72777191):

```bash
export OLLAMA_BASE_URL="http://127.0.0.1:11434"
```

@typpo fyi I think this is a bug here.

An aside is it would be good to have the `logger.debug` call log the complete [payload here](https://github.com/promptfoo/promptfoo/blob/da8c07643766e68dcc3744103c9225eb16ea3ee1/src/providers/ollama.ts#L51).

Here is my `/etc/hosts` file:

```txt
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1        localhost
255.255.255.255  broadcasthost
::1              localhost
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1        kubernetes.docker.internal
# End of section
```

---

The issue here could be IPv4 vs IPv6: https://superuser.com/a/906141

Sadly I am not super OS/networking knowledgeable

Perhaps there can be some new feature where `promptfoo` first tries with `localhost`, and if a connection isn't established, then it falls back onto `127.0.0.1`

Based on your error output, it appears that your network is configured to prioritize ipv6 under some circumstances, and ollama probably only listens on ipv4.  Seems like making it `127.0.0.1` by default should fix this.

I agree with what you're saying, and this further agrees with it:

```bash
> curl -X POST --header 'Content-Type: application/json' "http://[::1]:11434/api/generate" -d '{
  "model": "llama2:13b",
  "prompt": "Your first prompt goes here"
}'
curl: (7) Failed to connect to ::1 port 11434 after 5 ms: Couldn't connect to server
```

Made this to ask about it: https://github.com/jmorganca/ollama/issues/560

Perhaps promptfoo could expose some new env var `DEFAULT_DOMAIN` as `localhost`, that way it could apply to any provider using `localhost`

Okay it looks like Ollama now supports IPv6.  So configuring the Ollama server for IPv6: `export OLLAMA_HOST=":11434" ./ollama serve`, `promptfoo` using `localhost` now works fine.  Confirmed now it was an IPv4 vs IPv6 issue.

@typpo feel free to not adjust the default domain for Ollama to 127.0.0.1 and leave as localhost, the fix is up to you.  An alternate route is to document IPv4 vs IPv6 configuration in the Ollama provider's docs.

One question is the above referenced commit [0d218fc](https://github.com/promptfoo/promptfoo/commit/0d218fc4389a284cced329912bc557afc4890420), I don't think it's on `main`, so it wasn't in the 0.24.1 release (fyi)

I'm going to close this - it sounds like it was really more of a problem on the ollama side, which was resolved.

I will open a docs PR tomorrow to at least document this potential pitfall