Repository: promptfoo/promptfoo
Issue: Exit code is blank on success (rather than 0)
Root cause: Documentation Deficiency

**Describe the bug**

I have a hello world system I am playing with to set up promptfoo as part of an automated evaluation pipeline and when it fails I get an exit code of 100 (expected) but when it succeeds I get an exit code of nothing (as opposed to 0).

As I was using the exit code to handle the next step in the pipeline my bash script fell over.

I believe this is unexpected behaviour and can find no mention of it in any other issues or PRs.

**To Reproduce**

Config (YAML):

```yaml
description: "Hello World PromptFoo Test"

prompts:
  - 'Hello, {{name}}! How are you today?'

providers:
  - echo

tests:
  - vars:
      name: "World"
      assert:
        - type: contains
          value: "Hello, World"
  - vars:
      name: "Alice"
      assert:
        - type: contains
          value: "Hello, Alice"

```

Steps to reproduce the behavior, including example Promptfoo configurations if applicable:

1. Run `npx promptfoo eval || EXIT_CODE=$?`
2. Get 2 successes, 0 failures, 0 errors
3. `echo $EXIT_CODE`
4. See nothingness

**Expected behavior**

Step 4 should return `0`

**System information:**

```json
{
  "version": "0.115.0",
  "platform": {
    "os": "linux",
    "release": "6.14.7-arch2-1",
    "arch": "x64",
    "nodeVersion": "v22.15.0"
  },
  "env": {},
  "configInfo": {
    "configExists": false,
    "configContent": null
  }
}
```

Promptfoo (or any executable) always has an exit code upon termination,
this is an issue related to your bash :)

I'm not a bash expert but your variable assignment is probably resetting
the exit code.  Try:

npx promptfoo eval
EXIT_CODE=$?
echo $EXIT_CODE


On Thu, Jun 19, 2025 at 5:41 AM Chas Nelson ***@***.***>
wrote:

> *ChasNelson1990* created an issue (promptfoo/promptfoo#4567)
> <https://github.com/promptfoo/promptfoo/issues/4567>
>
> *Describe the bug*
>
> I have a hello world system I am playing with to set up promptfoo as part
> of an automated evaluation pipeline and when it fails I get an exit code of
> 100 (expected) but when it succeeds I get an exit code of nothing (as
> opposed to 0).
>
> As I was using the exit code to handle the next step in the pipeline my
> bash script fell over.
>
> I believe this is unexpected behaviour and can find no mention of it in
> any other issues or PRs.
>
> *To Reproduce*
>
> Config (YAML):
>
> description: "Hello World PromptFoo Test"
> prompts:
>   - 'Hello, {{name}}! How are you today?'
> providers:
>   - echo
> tests:
>   - vars:
>       name: "World"
>       assert:
>         - type: contains
>           value: "Hello, World"
>   - vars:
>       name: "Alice"
>       assert:
>         - type: contains
>           value: "Hello, Alice"
>
> Steps to reproduce the behavior, including example Promptfoo
> configurations if applicable:
>
>    1. Run npx promptfoo eval || EXIT_CODE=$?
>    2. Get 2 successes, 0 failures, 0 errors
>    3. echo $EXIT_CODE
>    4. See nothingness
>
> *Expected behavior*
>
> Step 4 should return 0
>
> *System information:*
>
> {
>   "version": "0.115.0",
>   "platform": {
>     "os": "linux",
>     "release": "6.14.7-arch2-1",
>     "arch": "x64",
>     "nodeVersion": "v22.15.0"
>   },
>   "env": {},
>   "configInfo": {
>     "configExists": false,
>     "configContent": null
>   }
> }
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/4567>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJW2UINZGD2YKC54P6T3EKVYXAVCNFSM6AAAAAB7V2Q2JCVHI2DSMVQWIX3LMV43ASLTON2WKOZTGE3DAMRQGM3TQMY>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


Whoooooops.... yup... should have been `&&` for success not `||` (which only runs on failure)... apols for the noise! 🤦