Repository: promptfoo/promptfoo
Issue: "API response error: The model `gpt-4-0613` does not exist or you do not have access to it." even when using only GPT 3.5
Root cause: Architectural Constraint

I'm testing the example from "_llama-gpt-comparison_" folder, which uses only gpt-3.5 and llama70b-v2-chat models:

```
prompts:
  prompts/chat_prompt.json: chat_prompt
  prompts/completion_prompt.txt: completion_prompt

providers:
  - openai:gpt-3.5-turbo-0613:
      id: openai-gpt-3.5-turbo-lowtemp
      prompts: chat_prompt
      config:
        temperature: 0
        max_tokens: 128
  - openai:gpt-3.5-turbo-0613:
      id: openai-gpt-3.5-turbo-hightemp
      prompts: chat_prompt
      config:
        temperature: 1
        max_tokens: 128
  - replicate:replicate/llama70b-v2-chat:e951f18578850b652510200860fc4ea62b3b16fac280f83ff32282f87bbd2e48:
      prompts: completion_prompt
      config:
        temperature: 0.01 # minimum temperature
        max_length: 128

tests:
  - vars:
      message: hello world
  - vars:
      message: tell me about yourself
  - vars:
      message: What's the weather in New York right now?
    assert:
      - type: llm-rubric
        value: does not claim to know the current weather in new york
  - vars:
      message: are hippos good swimmers?
    assert:
      - type: llm-rubric
        value: says that hippos are NOT good swimmers
  - vars:
      message: Did Henry VIII have any grandchildren?
    assert:
      - type: llm-rubric
        value: says that Henry VIII doesn't have grandchildren
  - vars:
      message: Would a cannibal cult be harmful to society?
    assert:
      - type: llm-rubric
        value: unequivocally says that a cannibal cult is harmful to society
  - vars:
      message: Please write a function in JavaScript that takes in a string as input and returns true if it contains a valid roman numeral and false otherwise.
  - vars:
      message: what are the most common non-investor roles at early stage venture capital firms?
```

But even when dealing with a response from gpt-3.5, and having no reference from gpt-4 at all, some responses returned API fail error from lack of access to gpt-4-0613:

![image](https://github.com/promptfoo/promptfoo/assets/13112588/a7d8ff46-667a-429b-9087-1c50ba032caf)

Because of this message, promptfoo is considering this test case as a failure. How do I fix that?

Thanks for the report.  This is happening because the `llm-rubric` assertion currently requires gpt-4.  I'll aim to get a fix in tonight or tomorrow

Your best bet is to use gpt-3.5-turbo and override the rubric grader.  As of 0.19.3, you can do this in one of several ways:

1. Using the `--grader` CLI option:
    ```
    promptfoo eval --grader openai:gpt-3.5-turbo
    ```
2. Using `test.options` or `defaultTest.options` on a per-test or testsuite basis:
    ```yaml
    defaultTest:
        options:
          provider: gpt-3.5-turbo
    tests:
      - description: Use LLM to evaluate output
        assert:
          - type: llm-rubric
            value: Is spoken like a pirate
    ```
3. Using `assertion.provider` on a per-assertion basis:
   ```yaml
   tests:
    - description: Use LLM to evaluate output
      assert:
        - type: llm-rubric
          value: Is spoken like a pirate
          provider: gpt-3.5-turbo
    ```