Repository: promptfoo/promptfoo
Issue: Eval Filter Failing not working
Root cause: Algorithmic Error

**Describe the bug**

I'm using `eval --filter-failing` to re-run only the failing tests. The tests executed does not make any sense given the filter.

**To Reproduce**
Steps to reproduce the behavior, including example Promptfoo configurations if applicable:

1. Run `promptfoo eval`. It returns _Successes: 49 Failures: 19 Errors: 6_
2. Run `promptfoo eval --filter-failing run_id_from_step_1`. This runs two tests. Of the two tests, one is from the Successes list and one if from the Failures list. This does not make sense. I expect it would run the 16 failed tests.

**Screenshots**

```
> promptfoo eval
Starting evaluation eval-1k8-2025-09-28T15:42:06
Running 74 test cases (up to 4 at a time)...
Evaluating [████████████████████████████████████████] 100% | 74/74 ...
...
===================================================================================================================================
✔ Evaluation complete. ID: eval-1k8-2025-09-28T15:42:06

» Run promptfoo view to use the local web viewer
» Do you want to share this with your team? Sign up for free at https://promptfoo.app
» This project needs your feedback. What's one thing we can improve? https://promptfoo.dev/feedback
===================================================================================================================================
Duration: 1m 29s (concurrency: 4)
Successes: 49
Failures: 19
Errors: 6
Pass Rate: 66.22%

>  promptfoo eval --filter-failing eval-1k8-2025-09-28T15:42:06
Starting evaluation eval-3CD-2025-09-28T15:44:38
Running 2 test cases (up to 4 at a time)...
Evaluating [████████████████████████████████████████] 100% | 2/2
...
```

**System information:**

If possible, please output the results of `promptfoo debug` and paste the output here.

```
 promptfoo debug
===================================================================================================================================
Promptfoo Debug Information
===================================================================================================================================
{
  "version": "0.118.10",
  "platform": {
    "os": "linux",
    "release": "6.16.7-200.fc42.x86_64",
    "arch": "x64",
    "nodeVersion": "v24.6.0"
  },
  "env": {},
  "configInfo": {
    "configExists": false,
    "configContent": null
  }
}
===================================================================================================================================
Please include this output when reporting issues on GitHub: https://github.com/promptfoo/promptfoo/issues
```

`promptfooconfig.yaml` contains tests and scenarios spread across 10 fails listed under `tests` and `scenarios`.

Hi @rvernica,

Sorry for the bug and for not closing this sooner - this was fixed in **PR #5770** on September 30, 2025 (2 days after you reported it), released in version 0.118.11.

The `--filter-failing` flag wasn't correctly filtering failing tests when re-running evals. This has been resolved - only failing tests are now re-run when the flag is used.

Please pull the latest version and the flag should work as expected:
```bash
promptfoo eval --filter-failing
```

If you notice any other filtering issues, feel free to comment or re-open.

See the [Command Line documentation](https://www.promptfoo.dev/docs/usage/command-line) for more info.

Thanks for helping us improve promptfoo!