Repository: promptfoo/promptfoo
Issue: llm rubric question
Root cause: Documentation Deficiency

Greetings,

when using the following config and then after a successful eval run I am not seeing scores when doing the view. Anything missing in this config, I been at this for days and reading the help doc and not making any progress

```
prompts:
  - file: translate.prompt

providers:
  - openai:gpt-4o-mini

datasets:
  - file: translations.csv

tests:
  - vars:
      source: "{{source}}"
      targetSentence: "{{targetSentence}}"
      lang: "{{lang}}"
    assert:
      - type: similarity
        value: "{{targetSentence}}"
        threshold: 0.75
        weight: 1
      - type: llm-rubric
        rubric: |
          You are a translation evaluator.
          Compare the model output with the reference "{{targetSentence}}".
          Score from 1 (poor) to 10 (perfect).
```

Hi!

I found the issue in your configuration.
The parameter name is incorrect – it should be `value` instead of `rubric`.

```
assert:
  - type: llm-rubric
    value: |
      Evaluate the output based on how funny it is.  Grade it on a scale of 0.0 to 1.0, where:
      Score of 0.1: Only a slight smile.
      Score of 0.5: Laughing out loud.
      Score of 1.0: Rolling on the floor laughing.

      Anything funny enough to be on SNL should pass, otherwise fail.
```
https://www.promptfoo.dev/docs/configuration/expected-outputs/model-graded/llm-rubric/


If you're using a editor with YAML schema support, you can add this line at the top of your config file to get helpful validation:
`# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json`

🤦 🤦 🤦  thanks so much. 🦆 another pair of eyes always are helpful. let me try that and see what happens.

@pokutuna does not seem to make any difference I am not seeing any result coming back showing the scores. not sure where else to look. thanks

Besides the issue I pointed out, your entire configuration seems to be incorrect.
Please read the documentation and check the YAML schema.

- To reference a file under prompts, you need to write it as `- file://translate.prompt`
- There is no `datasets` field allowed
- When debugging, use the `--verbose` option. It will show you what is being exchanged.
- Start with a simpler example and isolate the problem.

Hi @yosiasz,

Thanks for the question. Based on the discussion, the issues with your config were:

1. **Parameter name**: Should be `value:` not `rubric:` for llm-rubric assertions
2. **File references**: Should be `file://translate.json` (we support .json, .txt, .md formats)
3. **No datasets field**: Vars should be defined directly in `tests:`, not under a `datasets:` field

Here's the corrected structure:

```yaml
prompts:
  - file://translate.json

providers:
  - openai:gpt-5-nano

tests:
  - vars:
      source: "Hello"
      targetSentence: "Bonjour"
      lang: "French"
    assert:
      - type: similarity
        value: "{{targetSentence}}"
        threshold: 0.75
      - type: llm-rubric
        value: |
          You are a translation evaluator.
          Compare the model output with the reference "{{targetSentence}}".
          Score from 1 (poor) to 10 (perfect).
```

As @pokutuna suggested, add this at the top of your config for schema validation:
```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
```

See the [llm-rubric documentation](https://www.promptfoo.dev/docs/configuration/expected-outputs/model-graded/llm-rubric) for more details.

If you're still having issues after fixing these, feel free to comment or re-open with your corrected config and the output from `promptfoo eval --verbose`.

Thanks\!