Repository: promptfoo/promptfoo
Issue: Request: suppress warning `Treating it as a text prompt.` from each `eval`
GitHub labels: Feature Request

Running `promptfoo==0.56.0` with this config:

```yaml
---
prompts: >-
  Extract the single letter answer from the following question and answer
  \n\nDoes protein encoded by TDRD1 have symmetrical or asymmetrical
  dimethylarginine modifications, or no dimethylarginine at all?
  \n\nOptions:
  A) No dimethylarginine
  B) Symmetrical dimethylarginine modifications
  C) Asymmetrical dimethylarginine modifications
  D) Insufficient information to answer this question
  \n\nAnswer:
tests:
    ...
```

```none
> npm run promptfoo -- eval

> promptfoo
> promptfoo eval

(node:75194) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Could not find prompt file: "Extract the single letter answer from the following question and answer \n\nDoes protein encoded by TDRD1 have symmetrical or asymmetrical dimethylarginine modifications, or no dimethylarginine at all? \n\nOptions: A) No dimethylarginine B) Symmetrical dimethylarginine modifications C) Asymmetrical dimethylarginine modifications D) Insufficient information to answer this question \n\nAnswer:". Treating it as a text prompt.
...
```

Specifically the warning `Could not find prompt file: "...". Treating it as a text prompt.` It shows up every time I invoke `promptfoo eval` despite my config being valid.

Is there some way to suppress that warning?

There isn't a way to suppress the warning, but I noticed something strange about your prompt.  Are you intentionally including the literal string `"\n"`?  If not you probably should use literal newlines (this [stackoverflow post](https://stackoverflow.com/a/21699210/219604) is useful).  Incidentally that will prevent the "maybe filepath" trigger that is causing the output you want to hide.

Yeah actually I did also observe that the `\n` was involved with the warning showing up or not showing up. Though I chose to ignore this seeming edge case in the warning not appearing because I liked when the warning didn't occur 😆.

Also in https://github.com/promptfoo/promptfoo/issues/729 we observed `\n` doesn't really play nice with JSON conversions, so actually I removed the `\n` from my prompt in favor of actual whitespace lines.

---

Feel free to use this issue as an opportunity to:

- Resolve the edge case where the warning doesn't occur, when perhaps it should
- Create a way to suppress this warning

Or just close this one out, as you stated there's not a way to suppress the warning 👍 

This warning is indeed a bit troublesome.

My situation is that I need to load the contents of a JSON file myself as prompt input (ref: issue [#680](https://github.com/promptfoo/promptfoo/issues/680)), and this warning will occur as long as there is any '\n' character in a string.

Illustration:
`Could not find prompt file: "[{"role":"user","content":"Line 1\nLine 2\n..."}]". Treating it as a text prompt.`

---

But I'm not sure if adding a rule `!str.includes('\\n')` is appropriate. After all, a file path can also match this pattern... 😭

---

### Workaround:
Appending the '\n' character to the end of the stringified JSON content can bypass this warning.

This has been fixed for a second time in https://github.com/promptfoo/promptfoo/pull/1084 It was released as 0.6.3 Please feel free to give that a try and report back if something is a miss,