Repository: promptfoo/promptfoo
Issue: Different prompts for the same model didn't work
Root cause: Documentation Deficiency

**Summary:**
I am experiencing an issue where two different prompts (base_prompt and pm_prompt) are being assigned to the same model provider (`ollama:qwen2.5:0.5b`), but I am seeing the same output for both. I expect the prompts to differ and show different results, but they appear identical.

**Steps to Reproduce:**
1. In the `promptfooconfig.yaml`, configure two different prompts: `base_prompt` and `pm_prompt`.
2. Assign both prompts to the same model provider `ollama:qwen2.5:0.5b`.
3. Observe that both prompts result in the same output despite being different.

**Expected Behavior:**
The model should produce different outputs based on the assigned prompts, one with `base_prompt` and the other with `pm_prompt`.

**Actual Behavior:**
Both prompts seem to trigger the same prompt despite being different in the configuration.
![image](https://github.com/user-attachments/assets/57debc9f-0b3a-4d7f-bc29-aa961d257dcc)


**Additional Context:**
- The provider ID for both prompts is the same (`ollama:qwen2.5:0.5b`).
- The configuration is as follows:

```yaml
prompts:
  - id: llama_prompt.txt
    label: base_prompt
  - id: pm_prompt.txt
    label: pm_prompt

providers:
  - id: ollama:qwen2.5:0.5b
    prompts:
      - base_prompt
  - id: ollama:qwen2.5:0.5b
    prompts:
      - pm_prompt


> **Summary:** I am experiencing an issue where two different prompts (base_prompt and pm_prompt) are being assigned to the same model provider (`ollama:qwen2.5:0.5b`), but I am seeing the same output for both. I expect the prompts to differ and show different results, but they appear identical.
> 
> **Steps to Reproduce:**
> 
> 1. In the `promptfooconfig.yaml`, configure two different prompts: `base_prompt` and `pm_prompt`.
> 2. Assign both prompts to the same model provider `ollama:qwen2.5:0.5b`.
> 3. Observe that both prompts result in the same output despite being different.
> 
> **Expected Behavior:** The model should produce different outputs based on the assigned prompts, one with `base_prompt` and the other with `pm_prompt`.
> 
> **Actual Behavior:** Both prompts seem to trigger the same prompt despite being different in the configuration. ![image](https://private-user-images.githubusercontent.com/93692919/398520340-57debc9f-0b3a-4d7f-bc29-aa961d257dcc.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzUxMTI4NzUsIm5iZiI6MTczNTExMjU3NSwicGF0aCI6Ii85MzY5MjkxOS8zOTg1MjAzNDAtNTdkZWJjOWYtMGIzYS00ZDdmLWJjMjktYWE5NjFkMjU3ZGNjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDEyMjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMjI1VDA3NDI1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWJmMjYyOWE1NzM0YmQzMzQ3YWZlMDJlNzA1Yzg3NGNkOTgxZWNiZmIzMGNhYTg1ZmJjZWNkMmI4MWFmNDY5NzUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.DutYkkg427Yqwp21wL6Ro6wARMR6NIwRikjoULPZM1c)
> 
> **Additional Context:**
> 
> * The provider ID for both prompts is the same (`ollama:qwen2.5:0.5b`).
> * The configuration is as follows:
> 
> ```yaml
> prompts:
>   - id: llama_prompt.txt
>     label: base_prompt
>   - id: pm_prompt.txt
>     label: pm_prompt
> 
> providers:
>   - id: ollama:qwen2.5:0.5b
>     prompts:
>       - base_prompt
>   - id: ollama:qwen2.5:0.5b
>     prompts:
>       - pm_prompt
> ```

I tried the following code and it seems to work.  I noticed that the evaluation output displays two separate results.
```yaml
prompts: 
  - pm_prompt.txt
  - llama_prompt.txt

providers:
  - ollama:qwen2.5:0.5b

```


Hi @Lynn401,

Thanks for working through this and finding the solution!

As you discovered, the simplest way to test different prompts with the same model is:

```yaml
prompts:
  - pm_prompt.txt
  - llama_prompt.txt

providers:
  - ollama:qwen2.5:0.5b
```

This automatically tests both prompts against the model and displays them as separate columns. Much simpler than the complex provider-level mappings you were trying initially!

See the [Configuration Guide](https://www.promptfoo.dev/docs/configuration/guide) and [Parameters](https://www.promptfoo.dev/docs/configuration/parameters) for more info.

Thanks for sharing your solution!