Repository: promptfoo/promptfoo
Issue: Error while running npx promptfoo@latest init --no-interactive my-redteam-project on Mac M1
Root cause: Validation Gap

```
npx promptfoo@latest generate redteam -w                     
(node:18610) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Cache is disabled.
Synthesizing test cases for 2 prompts...
Plugins: contracts, excessive-agency, hallucination, harmful, hijacking, jailbreak, overreliance, pii, politics, prompt-injection, harmful:violent-crime, harmful:non-violent-crime, harmful:sex-crime, harmful:
 █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 2% | ETA: 1s | 2/100/Users/me/.npm/_npx/81bbc6515d992ace/node_modules/tiny-invariant/dist/tiny-invariant.cjs.js:14
    throw new Error(value);
          ^

Error: Invariant failed: Expected purpose to be a string
    at invariant (/Users/me/.npm/_npx/81bbc6515d992ace/node_modules/tiny-invariant/dist/tiny-invariant.cjs.js:14:11)
    at getPurpose (/Users/me/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/redteam/index.js:145:34)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async synthesize (/Users/me/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/redteam/index.js:83:41)
    at async Command.<anonymous> (/Users/me/.npm/_npx/81bbc6515d992ace/node_modules/promptfoo/dist/src/main.js:413:30)
```

Hi @rrbanda,

Thanks for reporting this issue. You can override the inferred purpose by setting the flag `--purpose 'your purpose'`. See [src/main.ts#L505](https://github.com/promptfoo/promptfoo/blob/main/src/main.ts#L505) for the CLI args, [src/redteam/index.ts#L120](https://github.com/promptfoo/promptfoo/blob/main/src/redteam/index.ts#L120) for where the override is applied, and [src/redteam/index.ts#L181](https://github.com/promptfoo/promptfoo/blob/main/src/redteam/index.ts#L181) for how the purpose is inferred from your prompts (and example purposes - consider inserting a debug log here). 

If you can share more of your config or would like to jump on a call to debug together, I would be happy to look at this with you. 

Also you may have to set OPENAI_API_KEY as an environment variable.

On Sun, Jun 23, 2024, 2:07 PM Michael ***@***.***> wrote:

> Hi @rrbanda <https://github.com/rrbanda>,
>
> Thanks for reporting this issue. You can override the inferred purpose by
> setting the flag --purpose 'your purpose'. See src/main.ts#L505
> <https://github.com/promptfoo/promptfoo/blob/main/src/main.ts#L505> for
> the CLI args, src/redteam/index.ts#L120
> <https://github.com/promptfoo/promptfoo/blob/main/src/redteam/index.ts#L120>
> for where the override is applied, and src/redteam/index.ts#L181
> <https://github.com/promptfoo/promptfoo/blob/main/src/redteam/index.ts#L181>
> for how the purpose is inferred from your prompts (and example purposes -
> consider inserting a debug log here).
>
> If you can share more of your config or would like to jump on a call to
> debug together, I would be happy to look at this with you.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/promptfoo/promptfoo/issues/1001#issuecomment-2185323665>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACLYJWFMPPDQBYZ4JFNLADZI42JVAVCNFSM6AAAAABJYUPT6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGMZDGNRWGU>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>


@typpo basically I would like to be able to generate using ollama/mistral

Got it. @mldangelo is working on this in #1002 

Thanks @typpo looks this is about to be merged @mldangelo 

@rrbanda We just released the ability to configure custom providers in `0.66.0`. You can try it out with (for example)
```sh
npx promptfoo@latest generate redteam -w --provider ollama:llama2-uncensored
```
I also added better debug logging for when `purpose` is not a string. Please give it a try and let me know if you run into any issues. 

Thanks @mldangelo i m working on evaluating this . will keep posted if something comes up