Repository: promptfoo/promptfoo
Issue: Templating with environment variable reference is not being resolved
Root cause: Documentation Deficiency

**Describe the bug**
_A clear and concise description of what the bug is._

Trying to follow the guide on [Accessing environment variables](https://www.promptfoo.dev/docs/configuration/guide/#accessing-environment-variables), I am finding that I am not being able to run an evaluation that runs with a templated environment variable.

**To Reproduce**
_Steps to reproduce the behavior, including example Promptfoo configurations if applicable:_

1. Set this minimal `promptfooconfig.yaml`

```yaml
providers:
- openai:gpt-4o

prompts:
  - 'Write {{ headline }}'

tests:
  - vars:
      headline: 'Articles about {{ env.TOPIC }}'
```

2. Set your environment variables either with a dotenv file (.env) or with `export` commands. For example: `export TOPIC=cars`
3. Run `promptfoo eval -o result.json`
4. See the output table:

```console
┌──────────────────────────────────────┬──────────────────────────────────────┐
│ headline                             │ [openai:gpt-4o] Write {{ headline }} │
├──────────────────────────────────────┼──────────────────────────────────────┤
│ Articles about {{ env.TOPIC }}       │ [PASS] It seems there is a           │
│                                      │ placeholder for the topic in your    │
│                                      │ request. Please provide the specific │
│                                      │ topic you'd like me to write         │
│                                      │ articles about, and I'll be happy to │
│                                      │ assist you!                          │
└──────────────────────────────────────┴──────────────────────────────────────┘
```

<details>

<summary>Full result.json outpu</summary>

```json
{
  "evalId": "eval-5zZ-2025-02-17T16:28:58",
  "results": {
    "version": 3,
    "timestamp": "2025-02-17T16:28:58.547Z",
    "prompts": [
      {
        "raw": "Write {{ headline }}",
        "label": "Write {{ headline }}",
        "id": "550a9577f78d2cfca26100ca1d7f0e86e92cc22ca37e1783482205c8910b1988",
        "provider": "openai:gpt-4o",
        "metrics": {
          "score": 1,
          "testPassCount": 1,
          "testFailCount": 0,
          "testErrorCount": 0,
          "assertPassCount": 0,
          "assertFailCount": 0,
          "totalLatencyMs": 4,
          "tokenUsage": {
            "total": 50,
            "prompt": 0,
            "completion": 0,
            "cached": 50,
            "numRequests": 1,
            "completionDetails": {
              "reasoning": 0,
              "acceptedPrediction": 0,
              "rejectedPrediction": 0
            }
          },
          "namedScores": {},
          "namedScoresCount": {},
          "cost": 0.00038750000000000004
        }
      }
    ],
    "results": [
      {
        "cost": 0.00038750000000000004,
        "gradingResult": {
          "pass": true,
          "score": 1,
          "reason": "No assertions",
          "tokensUsed": {
            "total": 0,
            "prompt": 0,
            "completion": 0,
            "cached": 0
          },
          "assertion": null
        },
        "id": "b1e2bdd3-db28-4c0f-86a0-8c9709fd78a8",
        "latencyMs": 4,
        "namedScores": {},
        "prompt": {
          "raw": "Write Articles about {{ env.TOPIC }}",
          "label": "Write {{ headline }}"
        },
        "promptId": "550a9577f78d2cfca26100ca1d7f0e86e92cc22ca37e1783482205c8910b1988",
        "promptIdx": 0,
        "provider": {
          "id": "openai:gpt-4o",
          "label": ""
        },
        "response": {
          "output": "It seems there is a placeholder for the topic in your request. Please provide the specific topic you'd like me to write articles about, and I'll be happy to assist you!",
          "tokenUsage": {
            "cached": 50,
            "total": 50
          },
          "cached": true,
          "cost": 0.00038750000000000004
        },
        "score": 1,
        "success": true,
        "testCase": {
          "vars": {
            "headline": "Articles about {{ env.TOPIC }}"
          },
          "assert": [],
          "options": {},
          "metadata": {}
        },
        "testIdx": 0,
        "vars": {
          "headline": "Articles about {{ env.TOPIC }}"
        },
        "metadata": {},
        "failureReason": 0
      }
    ],
    "stats": {
      "successes": 1,
      "failures": 0,
      "errors": 0,
      "tokenUsage": {
        "cached": 50,
        "completion": 0,
        "prompt": 0,
        "total": 50,
        "numRequests": 1,
        "completionDetails": {
          "reasoning": 0,
          "acceptedPrediction": 0,
          "rejectedPrediction": 0
        }
      }
    }
  },
  "config": {
    "prompts": [
      "Write {{ headline }}"
    ],
    "providers": [
      "openai:gpt-4o"
    ],
    "tests": [
      {
        "vars": {
          "headline": "Articles about {{ env.TOPIC }}"
        }
      }
    ],
    "sharing": true,
    "outputPath": [
      "result.json"
    ],
    "extensions": []
  },
  "shareableUrl": null
}
```
</details>

**Expected behavior**
_A clear and concise description of what you expected to happen._

The `{{ ENV.<variable> }}` reference should have been replaced in the prompt.


**System information:**

_If possible, please output the results of `promptfoo debug` and paste the output here._

<details>
<summary>`promptfoo debug` output</summary>
```json
{
  "version": "0.104.4",
  "platform": {
    "os": "darwin",
    "release": "24.3.0",
    "arch": "arm64",
    "nodeVersion": "v22.13.1"
  },
  "env": {},
  "configInfo": {
    "defaultConfigPath": "/Users/pablosanjose/projects/tmp/promptfoo-debug/promptfooconfig.yaml",
    "configExists": true,
    "configContent": {
      "config": {
        "tags": {},
        "prompts": [
          "Write {{ headline }}"
        ],
        "providers": [
          "openai:gpt-4o"
        ],
        "tests": [
          {
            "vars": {
              "headline": "Articles about {{ env.TOPIC }}"
            }
          }
        ],
        "scenarios": [],
        "env": {},
        "sharing": true,
        "defaultTest": {
          "vars": {},
          "assert": [],
          "options": {},
          "metadata": {}
        },
        "outputPath": [],
        "extensions": [],
        "metadata": {}
      },
      "testSuite": {
        "tags": {},
        "prompts": [
          {
            "raw": "Write {{ headline }}",
            "label": "Write {{ headline }}"
          }
        ],
        "providers": [
          {
            "env": {},
            "modelName": "gpt-4o",
            "config": {
              "basePath": "/Users/pablosanjose/projects/tmp/promptfoo-debug"
            },
            "label": ""
          }
        ],
        "providerPromptMap": {},
        "tests": [
          {
            "vars": {
              "headline": "Articles about {{ env.TOPIC }}"
            }
          }
        ],
        "scenarios": [],
        "defaultTest": {
          "metadata": {},
          "options": {},
          "vars": {},
          "assert": []
        },
        "nunjucksFilters": {},
        "extensions": []
      },
      "basePath": "/Users/pablosanjose/projects/tmp/promptfoo-debug"
    }
  }
}
```
</details>

**Additional context**
_None_


Hi @pablosjv! 👋 Thank you for reporting this issue and providing such a detailed reproduction. You're right that the environment variable substitution isn't working in your configuration.The issue is that environment variable substitution (`{{ env.TOPIC }}`) currently only works in prompt templates, not in the `vars` section. For example you can do this:

```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - echo
prompts:
  - 'Write articles about {{ env.TOPIC }}'
tests:
  - vars: {}
```
with 
```bash
TOPIC=dinosaurs promptfoo eval
```

We can consider adding environment variable support in the `vars` section in a future release but may choose not to given how easy it is to leak secrets via env variables. If you need a workaround, please see if you can move your env var substitution into your prompt. If that doesn't work for your use case, let us know. 

Thanks for the clarification, @mldangelo. And for the quick response!

I will try to play around and see if I can adapt the setup to use directly the prompt var. I will let you know how it goes 😄 

The example in the docs is using the vars for this. This is the example ([source](https://www.promptfoo.dev/docs/configuration/guide/#accessing-environment-variables)):

```
tests:
  - vars:
      headline: 'Articles about {{ env.TOPIC }}'
```

Should the documentation be updated? I am not familiar with the project structure, but I can try to contribute to that. I guess the change will be [here](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/guide.md?plain=1#L533-L541)