Repository: promptfoo/promptfoo
Issue: Redteam language config
Root cause: Documentation Deficiency

I follow the docs of redteam plugins to set the language of generated prompts to Chinese as follows:
```
# content in promptfooconfig.yaml
# target
redteam:
  purpose: ''
  plugins:
    - id: harmful:hate  # Tests handling of hate speech and discriminatory content
    - id: harmful:self-harm  # Tests handling of self-harm and suicide content
  strategies:
    - id: basic  # Original plugin tests without any additional strategies or optimization
  language: 'Chinese'
# other config
```
however, when generation and test ends, prompts in `redteam.yaml` and the answer of target in reports are still English. How can I handle with the situation?

Hi @adwur, language has to be set on a per-plugin basis.  See https://www.promptfoo.dev/docs/red-team/plugins/#setting-languages OR see multilingual plugin docs https://www.promptfoo.dev/docs/red-team/strategies/multilingual/

Hi @typpo, thank you for your response. Regarding the first link you provided, I had previously tried setting the language on a per-plugin basis. The file content was as follows: 

```
# promptfooconfig.yaml
targets:
  - id: deepseek:deepseek-chat
    label: ds_v3_chinese
    config:
      temperature: 1.0
      apiKey: xxx
prompts:
  - '{{prompt}}'
redteam:
  purpose: ''
  # language: 'Chinese'
  plugins:
    - id: harmful:hate  # Tests handling of hate speech and discriminatory content
      config:
        modifiers:
          language: 'Chinese'
    - id: harmful:self-harm  # Tests handling of self-harm and suicide content
      config:
        modifiers:
          language: 'Chinese'
  strategies:
    - id: basic  # Original plugin tests without any additional strategies or optimizations
defaultTest:
  options:
    transformVars: '{ ...vars, sessionId: context.uuid }'
```

However, the prompt examples in the generated `redteam.yaml` were still in English.

Besides, I had tried several other ways shown in https://www.promptfoo.dev/docs/red-team/configuration/#configuration-structure , https://www.promptfoo.dev/docs/red-team/configuration/#language , and https://www.promptfoo.dev/docs/red-team/configuration/#language-1 , but all of these links don't work.

For the second link, I modified the strategy content according to the link as follows:

```
redteam:
  purpose: ''
  # language: 'Chinese'
  plugins:
    - id: harmful:hate  # Tests handling of hate speech and discriminatory content
    - id: harmful:self-harm  # Tests handling of self-harm and suicide content
  strategies:
    - id: multilingual
      config:
        languages:
          - zh-CN
    - id: basic  # Original plugin tests without any additional strategies or optimizations
      config:
        enabled: false
defaultTest:
  options:
    transformVars: '{ ...vars, sessionId: context.uuid }'
```

but I encountered an error when executing `promptfoo redteam generate`:

```
Test Generation Report:
┌─────┬──────────┬────────────────────────────────────────┬────────────┬────────────┬──────────────┐
│ #   │ Type     │ ID                                     │ Requested  │ Generated  │ Status       │
├─────┼──────────┼────────────────────────────────────────┼────────────┼────────────┼──────────────┤
│ 1   │ Plugin   │ harmful:hate                           │ 5          │ 5          │ Success      │
├─────┼──────────┼────────────────────────────────────────┼────────────┼────────────┼──────────────┤
│ 2   │ Plugin   │ harmful:self-harm                      │ 5          │ 5          │ Success      │
├─────┼──────────┼────────────────────────────────────────┼────────────┼────────────┼──────────────┤
│ 3   │ Strategy │ multilingual                           │ 0          │ 0          │ Failed       │
└─────┴──────────┴────────────────────────────────────────┴────────────┴────────────┴──────────────┘
No test cases generated. Please check for errors and try again.
```

Could you please help me determine where the issue might be? Thank you!

Hi @adwur,

1. The issue was in how the language was configured. Instead of using 'Chinese', we need to:
   - Use the ISO code `zh-CN`
   - Configure it at both plugin and strategy levels
   - Keep the basic strategy enabled (don't set `enabled: false`)

Here's the working configuration:

```yaml
redteam:
  plugins:
    - id: harmful:hate
      config:
        modifiers:
          language: zh-CN    
    - id: harmful:self-harm
      config:
        modifiers:
          language: zh-CN
  strategies:
    - id: multilingual
      config:
        languages:
          - zh-CN
```

After applying these changes, you should see both English and Chinese test cases in your redteam.yaml file, with proper translations and metadata for each language variant.



Hi @AISimplyExplained, thanks for your help. I had tried several times and got the working configuration - I set the strategy on redteam setup web UI:

![Image](https://github.com/user-attachments/assets/c1445072-568a-41ae-8ed5-8df0197ad94e)

after that configuration in `promptfooconfig.yaml` is:

```yaml
redteam:
  plugins:
    - id: harmful:hate
    - id: harmful:self-harm
  strategies:
    - id: multilingual
      config:
        languages:
          - chinese
```
The configuration can generate both English and Chinese test cases. Besides, I have tried the configuration you prodivied, and that also works and generates test cases as expected. Thanks again!

When using multilingual, there will still be test cases in English, and the assertion results will also be in English. May I ask how to fully use Chinese?