Repository: promptfoo/promptfoo
Issue: Test Provider Override is not working
Root cause: Configuration Error

**Describe the bug**
While passing provider for a specific test case, it should override the global providers but it is not working for me. Can you help?

**To Reproduce**
**Config file** - 
description: "Testing two-step LLM pipeline: Intent Detection and Task Execution"
providers:
  - id: openai:gpt-4o # This will not be used if providers are specified per test
    config:
      apiKey: <APIKEY>

prompts:
  - file://prompts/intent_prompt.txt

outputPath: ./results/output.json
evaluateOptions:
  maxConcurrency: 4
  delay: 1000
  showProgressBar: true

tests:
  - description: "Intent Detection Test Cases"
    vars:
      user_input: "sink is leaking"
  - provider:
      id: openai:gpt-4o-mini
      config:
        apiKey: <APIKEY>
        
        
**Prompt**
You are an experienced manager with linguistic background, responsible for identifying suitable #Type for user message based on below example #Type,#Description, #Utterances along with ##ResponseRules## below:\n              #Type: \"Property_Search\", #Utterances: I am looking to sell my home., I want a rental property., i want to buy a house, book an appointment, i want an appointment, i want to book a meeting, can you setup some time on calendar?, #Description: You are Tracy, a sales assistant working for DialGood Real Estate. Your purpose is to help with prospecting new clients who are looking to sell or buy a property and lead them to book an appointment with one of the team members.\n\n#Type: \"Tenant_Support_- Property Managers\", #Utterances: My sink is clogged and not draining properly., I need a plumber, The glass on the window is broken., Someone broke into my house and damaged it., i want an appointment, i want to book a meeting, can you setup some time on calendar?, #Description: You are a virtual assistant helping tenants of a rental property to report issues or ask questions. Your job is to assist tenants in specifying their problem and automatically categorize it based on the type of request. The categories are:\n\nMaintenance Requests: Issues like broken appliances, plumbing problems, electrical issues, HVAC problems, etc.\nPayment Inquiries: Questions about rent payments, security deposits, or payment extensions.\nLease Agreements: Inquiries about lease terms, renewals, or early termination.\nProperty Issues: Problems like pest infestations, noise disturbances, or shared amenity issues.\nMove-In/Move-Out: Questions about move-in or move-out procedures, inspections, or logistics.\nUtility Services: Issues with water, gas, electricity, or internet services.\nGeneral Queries: General questions about property rules, local services, etc.\nEmergency Situations: Urgent repair needs or safety concerns that require immediate attention.\n\n.\n              \n              ##ResponseRules##:\n               - Only respond with exact #Type value any nothing else. \n               - Do not assume any type. \n               - Use original name and spelling of the task when recommending. \n               - Ensure you reflect on user's request and do your best to match it with the most suitable #Type.\n               - If no type is suitable, respond only with 'None'.
User's message: "{{ user_input }}"
Intent:


**Expected behavior**
It should override the gpt-4o model and use gpt-4o-mini but that's not happening

**Screenshots**
![image](https://github.com/user-attachments/assets/41323243-6983-4ac4-8253-e362b8fb4dac)

- Promptfoo version: 0.103.3



Hi @jammy23495, thanks for the detailed report! The provider override is working, but the Web UI and output json don’t reflect it. You can verify it’s using the correct provider by running with the --verbose flag. I’ll update the Web UI and output to display the correct provider and will open a PR today. I’ll keep you posted here!

Thanks @mldangelo for the response. The response I shared is from CLI. Is it not showing in Command Line too?

Hi @jammy23495,  we released a fix for this issue in [0.103.4](https://github.com/promptfoo/promptfoo/releases/tag/0.103.4). Please update to the latest version and give it a try. If the problem persists or you want to handle overrides differently, feel free to let me know, and we can revisit the issue. Thanks for your patience and for helping us improve Promptfoo!