Repository: conan-io/conan
Pull request: proposing new --format=<format> --out-file=<file.ext> feature
Changelog: Feature: Allow defining ``--out-file=file.ext`` instead of ``--format=ext > file.ext`` to write to files directly and avoid issues with redirects.
Docs: https://github.com/conan-io/docs/pull/3966

This would solve issues with stdout/stderr redirects and potential pollution, while not using new arguments, not breaking existing code and allow easy introduction to existing formatters.

Close https://github.com/conan-io/conan/issues/17462

Problem statement:
[bug] Warnings from Python libraries pollute json output
### Describe the bug

Conan 2

### How to reproduce it

We're using the parameter `--format json` when running `conan create` to get some information about the build. The output is redirected into a file using `> buildinfo.json`. The problem is now that when an internally used library like urllib emits warnings to stdout this output ends up in the json file, making it invalid. We stumbled across this bug after updating to Python 3.11 and a new urllib version which now prints warnings whenever a conanfile performs a download with `verify=False`. 

My suggestion would be to change the `--format json` parameter to `--json <filename>` and write only the json data to the specified file.

Hints from issue discussion:
Hi @jasal82,

Thanks a lot for your question. 

I was trying to check the issue reproducing it but it seems that the logs from urlib3 are not polluting the stdout for me. 
We have specifically a line in Conan to avoid this here: https://github.com/conan-io/conan/blob/77a70515cb5f4057d6ff09ffdde4df779bf7d99f/conans/client/rest/conan_requester.py#L24 as suggested in the urlib3 docs: https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning

Could you please guide me to try to reproduce the issue?

Commit message: add out-file only if formatters

Patch to review:
diff --git a/conan/api/output.py b/conan/api/output.py
index 4d110270c56..1a9c0ffad1e 100644
--- a/conan/api/output.py
+++ b/conan/api/output.py
@@ -294,10 +294,7 @@ def cli_out_write(data, fg=None, bg=None, endline="\n", indentation=0):
         sys.stdout.write(data)
     else:
         data = f"{' ' * indentation}{data}{endline}"
-        # https://github.com/conan-io/conan/issues/17245 avoid colorama crash and overhead
-        colorama.deinit()
         sys.stdout.write(data)
-        colorama.reinit()
 
 
 class TimedOutput:
diff --git a/conan/cli/command.py b/conan/cli/command.py
index 551fab40eea..11afd34fe9d 100644
--- a/conan/cli/command.py
+++ b/conan/cli/command.py
@@ -1,5 +1,8 @@
 import argparse
 import textwrap
+from contextlib import redirect_stdout
+
+import colorama
 
 from conan.api.output import ConanOutput
 from conan.errors import ConanException
@@ -63,10 +66,13 @@ def _help_formatters(self):
         return [formatter for formatter in self._formatters if formatter != "text"]
 
     def _init_formatters(self, parser):
-        formatters = self._help_formatters
-        if formatters:
-            help_message = "Select the output format: {}".format(", ".join(formatters))
+        if self._help_formatters:
+            help_message = "Select the output format: {}".format(", ".join(self._help_formatters))
             parser.add_argument('-f', '--format', action=OnceArgument, help=help_message)
+        # Add the out-file argument if any formatters exist, even the default one not shown in help
+        if self._formatters:
+            parser.add_argument("--out-file", action=OnceArgument,
+                                help="Write the output of the command to the specified file instead of stdout.")
 
     @property
     def name(self):
@@ -83,11 +89,8 @@ def doc(self):
     def _format(self, parser, info, *args):
         parser_args, _ = parser.parse_known_args(*args)
 
-        default_format = "text"
-        try:
-            formatarg = parser_args.format or default_format
-        except AttributeError:
-            formatarg = default_format
+        formatarg = getattr(parser_args, "format", None) or "text"
+        out_file = getattr(parser_args, "out_file", None)
 
         try:
             formatter = self._formatters[formatarg]
@@ -95,7 +98,16 @@ def _format(self, parser, info, *args):
             raise ConanException("{} is not a known format. Supported formatters are: {}".format(
                 formatarg, ", ".join(self._help_formatters)))
 
-        formatter(info)
+        if out_file:
+            with open(out_file, 'w') as f:
+                with redirect_stdout(f):
+                    formatter(info)
+            ConanOutput().info(f"Formatted output saved to '{out_file}'")
+        else:
+            # https://github.com/conan-io/conan/issues/17245 avoid colorama crash and overhead
+            colorama.deinit()
+            formatter(info)
+            colorama.reinit()
 
     @staticmethod
     def _dispatch_errors(info):
diff --git a/test/integration/command_v2/test_output.py b/test/integration/command_v2/test_output.py
index 534f2c46525..52a6b960399 100644
--- a/test/integration/command_v2/test_output.py
+++ b/test/integration/command_v2/test_output.py
@@ -1,3 +1,5 @@
+import json
+
 from conan.test.assets.genconanfile import GenConanfile
 from conan.test.utils.tools import TestClient
 from conan.test.utils.env import environment_update
@@ -260,3 +262,32 @@ def test_exception_errors(self):
         t.run("create . -vquiet", assert_error=True)
         assert "ERROR: Tagged error" not in t.out
         assert "ConanException: Untagged error" not in t.out
+
+
+def test_formatter_redirection_to_file():
+    c = TestClient(light=True)
+    c.save({"conanfile.py":  GenConanfile("pkg", "0.1")})
+
+    c.run("config home --out-file=cmd_out.txt")
+    assert "Formatted output saved to 'cmd_out.txt'" in c.out
+    cmd_out = c.load("cmd_out.txt")
+    assert f"{c.cache_folder}" in cmd_out
+    assert not f"{c.cache_folder}" in c.stdout
+
+    c.run("graph info . --format=json --out-file=graph.json")
+    assert "Formatted output saved to 'graph.json'" in c.out
+    graph = json.loads(c.load("graph.json"))
+    assert len(graph["graph"]["nodes"]) == 1
+    assert not "nodes" in c.stdout
+
+    c.run("graph info . --format=html --out-file=graph.html")
+    assert "Formatted output saved to 'graph.html'" in c.out
+    html = c.load("graph.html")
+    assert "<head>" in html
+    assert not "<head>" in c.stdout
+
+    c.run("install . --format=json --out-file=graph.json")
+    assert "Formatted output saved to 'graph.json'" in c.out
+    graph = json.loads(c.load("graph.json"))
+    assert len(graph["graph"]["nodes"]) == 1
+    assert not "nodes" in c.stdout


Code review comments:
diff_hunk: @@ -63,10 +66,13 @@ def _help_formatters(self):
         return [formatter for formatter in self._formatters if formatter != "text"]
 
     def _init_formatters(self, parser):
-        formatters = self._help_formatters
-        if formatters:
-            help_message = "Select the output format: {}".format(", ".join(formatters))
+        if self._help_formatters:
+            help_message = "Select the output format: {}".format(", ".join(self._help_formatters))
             parser.add_argument('-f', '--format', action=OnceArgument, help=help_message)
+        # Add the out-file argument if any formatters exist, even the default one not shown in help
+        if self._formatters:
path: conan/cli/command.py
text: @author:
But this is always ``True``, as it is hardcoded to be ``self._formatters = {"text": lambda x: None}`` and it cannot be removed.
It seems the gap is not being able to differentiate when a user really defines the ``formatters={"text": myformatter`` and that default formatter that does nothing, which is not very clear the intention.

diff_hunk: @@ -83,19 +89,25 @@ def doc(self):
     def _format(self, parser, info, *args):
         parser_args, _ = parser.parse_known_args(*args)
 
-        default_format = "text"
-        try:
-            formatarg = parser_args.format or default_format
-        except AttributeError:
-            formatarg = default_format
+        formatarg = getattr(parser_args, "format", None) or "text"
+        out_file = getattr(parser_args, "out_file", None)
 
         try:
             formatter = self._formatters[formatarg]
         except KeyError:
             raise ConanException("{} is not a known format. Supported formatters are: {}".format(
                 formatarg, ", ".join(self._help_formatters)))
 
-        formatter(info)
+        if out_file:
+            with open(out_file, 'w') as f:
+                with redirect_stdout(f):
+                    formatter(info)
+            ConanOutput().info(f"Formatted output saved to '{out_file}'")
+        else:
+            # https://github.com/conan-io/conan/issues/17245 avoid colorama crash and overhead
+            colorama.deinit()
path: conan/cli/command.py
text: @author:
This will be problematic if some formatter uses color. Note the previous location was conditioned to not ``if (fg or bg) and _color_enabled(sys.stdout):``