Repository: iterative/dvc

Issue:
dvc: expand on `--no-pager` option
1. ~~Make it the default for `dvc exp show`?~~ No longer needed after #5906
2. Include it in `dvc dag`?
3. Maybe put it directly in `dvc --no-pager` so it works for any commands that need it in the future.
4. Any other existing commands that need it? E.g. `list` or `stage list`

Rel. #4060
dag: do not use pager when output fits into one screen
The same way as Git determines screen size and does not use "less" to show diff, log, etc we should avoid using it when it's not actually needed.

Using "pager" should be considered an unavoidable "evil" in cases when output is too large. 


Comments:
What should the behavior be for dags that are too wide to display in the user's terminal? Wrapping lines (or truncating it) eliminates the usefulness of the output entirely here, it's not the same as a table where we can merge/hide columns until the table is small enough to fit.
Should we do `--no-pager` as a default in both of the cases? I use `--no-pager` more often than the default one.
> What should the behavior be for dags that are too wide

Good Q. Maybe just truncate it like I think you are saying we do for tables?

> Should we do --no-pager as a default in both of the cases?

I also use `--no-pager` all the time haha. At least give it a short flag?

BTW `git` has `--no-pager` in the top command e.g. `git --no-pager diff`. And it has a setting to configure the default paging behavior/command, I think. More options to consider.
> More options to consider.

detect if you have enough space or not and apply pager if not? that's what git does on my machine, but as we discussed with Ruslan it's not clear how/why it is happening.
For most of my scenarios, it is never enough space for the whole table. But I still prefer `--no-pager`.
BTW I renamed this issue and rewrote the description to reflect what we're actually talking about. To summarize, I propose:

Step 1. Make it the default for `exp show`
Step 2. Put it directly in `dvc --no-pager` and enable if for `dag` too (but not as default).

Qs:
- How to handle long lines? I say truncate them for now.
- Any other commands that need it? E.g. `list`

> detect if you have enough space or not and apply pager if not? that's what git does on my machine, but as we discussed with Ruslan it's not clear how/why it is happening.

Seems like Git is using `less -FRX` by default to do this: https://git-scm.com/docs/git-config#Documentation/git-config.txt-corepager. It would be great to do the same in dvc and make this the default.

>     * How to handle long lines? I say truncate them for now.

👍 

>     * Any other commands that need it? E.g. `list`

Ideally, we can abstract and set all of this at the top level as part of the UI overhaul that's ongoing. Thought @skshetry?
> > detect if you have enough space or not and apply pager if not? that's what git does on my machine, but as we discussed with Ruslan it's not clear how/why it is happening.
> 
> Seems like Git is using `less -FRX` by default to do this: https://git-scm.com/docs/git-config#Documentation/git-config.txt-corepager. It would be great to do the same in dvc and make this the default.

good to finally know the cause 😄, anyway it adds support to my proposal https://github.com/iterative/dvc/issues/4060#issuecomment-769846201 ;-)

I'm working on keeping the behavior same as `git` as proposed on https://github.com/iterative/dvc/issues/4060#issuecomment-769846201. But very few people will be able to see this as most users will have `PAGER` or `LESS` set. 
> I'm working on keeping the behavior same as `git` as proposed on [#4060 (comment)](https://github.com/iterative/dvc/issues/4060#issuecomment-769846201).

It would be great to have similar to Git behavior. However, it is not limiting by --no-pager/less mode. In Git when I quite from the less mode (like `git log`) I still see the output. I would not need `--no-pager` option of `exp run` if this would be the case. But the current default mode clears out the output - that's the problem.

@skshetry could you please clarify what users will see after quitting the command in less mode?
@dmpetrov, well I have two ideas:

1. ~~Print the table in a pager mode and in a non-pager mode in stderr after the pager exits. This way, you will see a non-collapsed form in the table and collapsed form after it exits.~~ Looks like this messes up `redirections`.
2. `DVC_PAGER="less -FRSX~" dvc exp show` -> This way, if the content can fit within the screen (vertically and horizontally), no pager will be used.  Otherwise, the pager will be shown where the user's prompt was (without clearing the screen to the top). After that, the content will still be on the user's screen.

  Second one has a disadvantage that when the user scrolls, it will leave the empty patch in the user's terminal. 

thoughts @pmrowla?
> It would be great to have similar to Git behavior. However, it is not limiting by --no-pager/less mode. In Git when I quite from the less mode (like `git log`) I still see the output.

@dmpetrov 

This is not actually default Git behavior (when I quit from `git log` inside a pager I do not see the output). It sounds to me like something in your machine/environment is configuring the system pager to be `less -X`.

Can you run
```
echo $PAGER
```
in a terminal and see what it outputs?

---

IMO we should leave the existing defaults, and if the user wants the `-X` behavior, they can configure `DVC_PAGER` as needed, the same way it works with configuring `PAGER` for git.

If anything, we could maybe consider just respecting the system `PAGER` instead of using our own env var (so that we would behave consistently with git within a given user's environment).
You can disregard my last comment, I see where Dave linked the git docs w/the info about their defaults now.

Using the same `less -FRSX` defaults makes sense to me, but we should probably also consider checking `$PAGER` and `$LESS` in the same way that git does as well

@skshetry 
@pmrowla, `dvc exp show` needs `-R` to work properly. `PAGER` is usually set to `less` and `LESS` is already set up in my machine with `-R` (by ohmyzsh?), but we cannot say the same for everyone. So it feels like we cannot just honour `$LESS`.

I'd propose we detect if the pager is `less` (with `DVC_PAGER`, `PAGER` and `DEFAULT_PAGER` in order), and apply `-R` even if `$LESS` is specified (this should be applied without us doing anything and deviates from the behavior with `git`). And, if there's no `$LESS` set, we could use `less -FRSX`.

Otherwise we would need to disable the styles.
@shcheklein On my machines(linux and mac) git does use the pager even if it can fit log/diff on the screen. I suppose that you have some pager configured that does that. Mind sharing `$echo $PAGER` and `$ git config core.pager`, please? Also if pager is `less`, could you share `$ alias less`?
Also, IIRC you are using pycharm, are you running git log from it? If so, could you try doing that from regular terminal too?
Sure, @efiop :

```
√ Projects/dvc.org % alias less
?1 Projects/dvc.org % echo $PAGER

√ Projects/dvc.org % git config core.pager
?1 Projects/dvc.org %
```
For the record: GIT_PAGER is also empty.

For me that results in git always outputting straight to teminal. Not sure what is going on.
@efiop, @shcheklein, I suspect `less` is told not to paginate if output is less than a page. 

Try:
```sh
export LESS="-F -X $LESS"
git branch
```
But, by default, git/less always paginates.
@skshetry That makes sense, but not quite understand why `dvc dag` doesn't do that as well. Might be missing some logic in `dvc/utils/pager.py`, i suppose.
@skshetry not sure how to check this

```
√ build/example-get-started % echo $LESS

```

it is also empty for me. I didn't change any setting myself.

I check the default Mac OS terminal - same behavior. Even if there are only two stages in DAG it paginates.
Should we repurpose this issue to consider this to be an expected behaviour (by adding `-FX` below)?
https://github.com/iterative/dvc/blob/fcdb503b4eff1297e5c1c1ed18f5606fe780e481/dvc/utils/pager.py#L15-L17
@skshetry Sounds good!

Patch:
diff --git a/dvc/utils/pager.py b/dvc/utils/pager.py
--- a/dvc/utils/pager.py
+++ b/dvc/utils/pager.py
@@ -14,40 +14,80 @@
 
 
 DEFAULT_PAGER = "less"
-DEFAULT_PAGER_FORMATTED = (
-    f"{DEFAULT_PAGER} --chop-long-lines --clear-screen --RAW-CONTROL-CHARS"
-)
-
-
-def make_pager(cmd):
+LESS = "LESS"
+PAGER_ENV = "PAGER"
+
+
+def prepare_default_pager(
+    clear_screen: bool = False,
+    quit_if_one_screen: bool = True,
+    ansi_escapes: bool = True,
+    chop_long_lines: bool = True,
+    no_init: bool = True,
+    no_tilde: bool = False,
+) -> str:
+    args = [DEFAULT_PAGER]
+    if clear_screen:
+        args.append("--clear-screen")  # -c
+    if quit_if_one_screen:
+        args.append("--quit-if-one-screen")  # -F
+    if ansi_escapes:
+        args.append("--RAW-CONTROL-CHARS")  # -R
+    if chop_long_lines:
+        args.append("--chop-long-lines")  # -S
+    if no_init:
+        args.append("--no-init")  # -X
+    if no_tilde:
+        args.append("--tilde")  # -~
+
+    return " ".join(args)
+
+
+def make_pager(cmd=None):
     def _pager(text):
         return pydoc.tempfilepager(pydoc.plain(text), cmd)
 
-    return _pager
+    return _pager if cmd else pydoc.plainpager
 
 
 def find_pager():
     if not sys.stdout.isatty():
-        return pydoc.plainpager
-
-    env_pager = os.getenv(DVC_PAGER)
-    if env_pager:
-        return make_pager(env_pager)
-
-    if os.system(f"({DEFAULT_PAGER}) 2>{os.devnull}") == 0:
-        return make_pager(DEFAULT_PAGER_FORMATTED)
-
-    logger.warning(
-        "Unable to find `less` in the PATH. Check out "
-        "{} for more info.".format(
-            format_link("https://man.dvc.org/pipeline/show")
+        return None
+
+    # pylint: disable=redefined-outer-name
+    pager = os.getenv(DVC_PAGER)
+    if not pager:
+        pager = os.getenv(PAGER_ENV)
+    if not pager:
+        if os.system(f"({DEFAULT_PAGER}) 2>{os.devnull}") != 0:
+            logger.warning(
+                "Unable to find `less` in the PATH. Check out "
+                "{} for more info.".format(
+                    format_link("https://man.dvc.org/pipeline/show")
+                )
+            )
+        else:
+            pager = DEFAULT_PAGER
+
+    if pager == DEFAULT_PAGER:
+        # if pager is less (i.e. default), regardless of `$LESS`, apply `-RS`.
+        # `-R` is required to render ansi escape sequences for exp show
+        # and, `-S` is required for horizontal scrolling.
+        less_env = bool(os.getenv(LESS))
+        pager = prepare_default_pager(
+            ansi_escapes=True,
+            chop_long_lines=True,
+            quit_if_one_screen=not less_env,
+            no_init=not less_env,
         )
-    )
-    return pydoc.plainpager
+
+    return pager
 
 
 def pager(text: str) -> None:
-    find_pager()(text)
+    _pager = find_pager()
+    logger.trace(f"Using pager: '{_pager}'")  # type: ignore[attr-defined]
+    make_pager(_pager)(text)
 
 
 class DvcPager(Pager):


Test patch:
diff --git a/tests/unit/ui/test_pager.py b/tests/unit/ui/test_pager.py
new file mode 100644
--- /dev/null
+++ b/tests/unit/ui/test_pager.py
@@ -0,0 +1,99 @@
+import pytest
+
+from dvc.env import DVC_PAGER
+from dvc.utils.pager import (
+    DEFAULT_PAGER,
+    LESS,
+    PAGER_ENV,
+    find_pager,
+    make_pager,
+    pager,
+)
+
+
+@pytest.fixture(autouse=True)
+def clear_envs(monkeypatch):
+    monkeypatch.delenv(DVC_PAGER, raising=False)
+    monkeypatch.delenv(PAGER_ENV, raising=False)
+    monkeypatch.delenv(LESS, raising=False)
+
+
+def test_find_pager_when_not_isatty(mocker):
+    mocker.patch("sys.stdout.isatty", return_value=False)
+    assert find_pager() is None
+
+
+def test_find_pager_uses_custom_pager_when_dvc_pager_env_var_is_defined(
+    mocker, monkeypatch
+):
+    monkeypatch.setenv(DVC_PAGER, "my-pager")
+    mocker.patch("sys.stdout.isatty", return_value=True)
+
+    assert find_pager() == "my-pager"
+
+
+def test_find_pager_uses_custom_pager_when_pager_env_is_defined(
+    mocker, monkeypatch
+):
+    monkeypatch.setenv(PAGER_ENV, "my-pager")
+    mocker.patch("sys.stdout.isatty", return_value=True)
+
+    assert find_pager() == "my-pager"
+
+
+def test_find_pager_uses_default_pager_when_found(mocker):
+    mocker.patch("sys.stdout.isatty", return_value=True)
+    mocker.patch("os.system", return_value=0)
+
+    assert DEFAULT_PAGER in find_pager()
+
+
+def test_find_pager_fails_to_find_any_pager(mocker):
+    mocker.patch("os.system", return_value=1)
+    mocker.patch("sys.stdout.isatty", return_value=True)
+
+    assert find_pager() is None
+
+
+@pytest.mark.parametrize("env", [DVC_PAGER, PAGER_ENV, None])
+def test_dvc_sets_default_options_on_less_without_less_env(
+    mocker, monkeypatch, env
+):
+    if env:
+        monkeypatch.setenv(env, "less")
+    mocker.patch("sys.stdout.isatty", return_value=True)
+    mocker.patch("os.system", return_value=0)
+
+    assert find_pager() == (
+        "less --quit-if-one-screen --RAW-CONTROL-CHARS"
+        " --chop-long-lines --no-init"
+    )
+
+
+@pytest.mark.parametrize("env", [DVC_PAGER, PAGER_ENV, None])
+def test_dvc_sets_some_options_on_less_if_less_env_defined(
+    mocker, monkeypatch, env
+):
+    if env:
+        monkeypatch.setenv(env, "less")
+    mocker.patch("sys.stdout.isatty", return_value=True)
+    mocker.patch("os.system", return_value=0)
+    monkeypatch.setenv(LESS, "-R")
+
+    assert find_pager() == "less --RAW-CONTROL-CHARS --chop-long-lines"
+
+
+def test_make_pager_when_no_pager_found(mocker, monkeypatch):
+    assert make_pager(None).__name__ == "plainpager"
+
+
+def test_pager(mocker, monkeypatch):
+    monkeypatch.setenv(DVC_PAGER, "my-pager")
+    mocker.patch("sys.stdout.isatty", return_value=True)
+
+    m_make_pager = mocker.patch("dvc.utils.pager.make_pager")
+    _pager = m_make_pager.return_value = mocker.MagicMock()
+
+    pager("hello world")
+    m_make_pager.assert_called_once_with("my-pager")
+    _pager.assert_called_once_with("hello world")