Repository: promptfoo/promptfoo
Issue: How to specify a custom grading prompt? 
Root cause: Documentation Deficiency

I'm trying to get a llm-rubric assertion result to map to the resulting score. For example my prompt is something like:

"Grade the output by the following specifications, keeping track of the points scored:
- Did the output mention x? +1 point
- Did the output describe y? + 1 point
- Did the output ask to clarify z? +1 point

Output your response in the following JSON format:
{`pass`: bool, `score`: number, `reason`: string}"

I'd like to see my llm-scored points output in the score column.
Is there a way to achieve this behaviour? I've tried providing a custom grading prompt to the `rubricPrompt` field in my assertion with no success. (Not sure if I'm using that field correctly, can't find much documentation on it)


P.S
[Ran into this error which is currently not specifying the correct assertion type ](https://github.com/promptfoo/promptfoo/blob/9891f79a8463fa819a9ffe168815f2c58ee1c390/src/assertions.ts#L430C1-L430C1)

Hi @AZielinsky95,

I just published 0.23.1, which includes some changes to make this easier.  I used the prompt you provided in this new example:  https://github.com/promptfoo/promptfoo/blob/main/examples/custom-grading-prompt/promptfooconfig.yaml

The results view displays the graded score after the `PASS`.  Right now the score is hidden if it is 1.0 or 0.0 (indicating failure).
![image](https://github.com/promptfoo/promptfoo/assets/310310/7aa8fb58-afc4-4871-97e1-c7810ebd64e1)

There's definitely room for improvement in making prompt overriding more intuitive.  In this case, you'd need to know about the magic variable `{{output}}`.  I'm going to update the documentation to describe this in more detail.

Let me know how this works for you, and if you have any other suggestions.

Works great! Appreciate how quickly you addressed this! 🚀 

Upon running my other assertion on 0.23.1 that uses the default grading prompt, I noticed the web view failed to render and threw the following error `TypeError: Cannot read properties of null (reading 'toFixed'). `

I believe this is due to the score property missing from the grading result. I think this would be fixed if you simply fallback to what you had before: `parsed.pass ? 1 : 0`  in the case score isn't present in the llm response.

This is my response with missing score:
``` JSON
"assertion": {
      "type": "llm-rubric",
      "value": "Are questions formatted as 3 numbered points on separate lines?",
      "provider": "openai:gpt-3.5-turbo-0613"
    },
    "pass": true,
    "reason": "the questions are formatted as 3 numbered points on separate lines",
    "tokensUsed": {
      "total": 341,
      "prompt": 319,
      "completion": 22
}
```

Added a commit, which I think handles your case and a handful of others:
- If neither pass nor score are set, test passes by default and score is 1.0
- If pass is not set, any score > 0 is considered a pass
- If score is not set, return 1.0 or 0.0 based on pass/fail
- If reason is not set, set a placeholder based on pass

Let me know if you see any problems with this.  If not, I'll cut a patch release.

Fixed! Thanks