Repository: tqdm/tqdm (Python)

Issue:
Add fixed width bar `bar_width` and chop overflow on ncols
- added bar_width to initializer and format_meter
- added better ncols handling for overflows on l_bar, bar, r_bar
- added tests

This fixes #623
This fixes #690
This helps prevent the spillover seen in #630

This option (bar_width) prevents the bar portion of the output from being resized dynamically, and makes the r_bar portion dynamic (it gets chopped).  This addresses some of the concerns of #630 as well, but does not make an individual bar multi-line capable.  It just ensures you do not get multiple lines.

If the l_bar side is larger than ncols, you get ncols of l_bar.
If l_bar + bar is larger than ncols, you get l_bar + some of the bar.
If t_bar + bar + r_bar is larger than ncols, some of r_bar gets chopped.

Is there anything else I need to do for this to be accepted?

Thanks.

- [x] I have visited the [source website], and in particular
  read the [known issues]
- [x] I have searched through the [issue tracker] for duplicates
- [x] I have mentioned version numbers, operating system and
  environment, where applicable:
  ```python
  import tqdm, sys
  print(tqdm.__version__, sys.version, sys.platform)
  ```
- [x] If applicable, I have mentioned the relevant/related issue(s)

  [source website]: https://github.com/tqdm/tqdm/
  [known issues]: https://github.com/tqdm/tqdm/#faq-and-known-issues
  [issue tracker]: https://github.com/tqdm/tqdm/issues?q=

set_description causes ghosting with long text
I'm using set_description() with a filename, so I can easily show which file is currently being processed.

If a long filename comes along, I think it's becoming multiline, and then when the next file (that isn't long) comes up, it leaves a ghost trail on the screen, very similar to when you use print() instead of tqdm.write().

I have simply truncated my filenames to a fixed size, but in an ideal world it would be nice to have the option of truncating this automatically as you will know exactly how many characters there are left available on screen once everything else is rendered. I'm simply guessing.

Or, is this a rendering issue that can be fixed and leave the multiline implementation as is?


Hints:

sort of duplicate of #630 

Don't think there's a better solution than pre-formatting. For example:

`tqdm(..., bar_format="{desc:9.9s}: {percentage:3.0f}%|{bar}{r_bar}"`

if you want a consistent `desc` of length `9`.
I'm not entirely sure I agree it's a duplicate - but that's up to you.

However, the ghosting is definitely undesirable. If it temporarily becomes multiline - I can cope with that, in fact, I would prefer that - but I just don't expect to see a ghost left behind when it switches back. So either that needs looking at, or we need the option of automatic cropping of the text to the available space to keep it on one line. 

Off the top of my head, you could do the string formatting for the bar (minus description) to get its length, then compare to the terminal width, and format for a second time to add the description in. Obviously a little slower, but if it was an optional parameter, I would pick it. 

Alternatively, if there was a way of requesting the number of characters in the progress bar, I could set the description to (""), request the number of characters, and then set the description length appropriately.

Patch:
diff --git a/tqdm/std.py b/tqdm/std.py
--- a/tqdm/std.py
+++ b/tqdm/std.py
@@ -10,8 +10,8 @@
 from __future__ import absolute_import, division
 # compatibility functions and utilities
 from .utils import _supports_unicode, _environ_cols_wrapper, _range, _unich, \
-    _term_move_up, _unicode, WeakSet, _basestring, _OrderedDict, _text_width, \
-    Comparable, RE_ANSI, _is_ascii, FormatReplace, \
+    _term_move_up, _unicode, WeakSet, _basestring, _OrderedDict, \
+    Comparable, _is_ascii, FormatReplace, disp_len, disp_trim, \
     SimpleTextIOWrapper, CallbackIOWrapper
 from ._monitor import TMonitor
 # native libraries
@@ -479,12 +479,13 @@ def format_meter(n, total, elapsed, ncols=None, prefix='', ascii=False,
             # Formatting progress bar space available for bar's display
             full_bar = Bar(
                 frac,
-                max(1, ncols - _text_width(RE_ANSI.sub('', nobar)))
+                max(1, ncols - disp_len(nobar))
                 if ncols else 10,
                 charset=Bar.ASCII if ascii is True else ascii or Bar.UTF)
             if not _is_ascii(full_bar.charset) and _is_ascii(bar_format):
                 bar_format = _unicode(bar_format)
-            return bar_format.format(bar=full_bar, **format_dict)
+            res = bar_format.format(bar=full_bar, **format_dict)
+            return disp_trim(res, ncols) if ncols else res
 
         elif bar_format:
             # user-specified bar_format but no total
@@ -496,10 +497,11 @@ def format_meter(n, total, elapsed, ncols=None, prefix='', ascii=False,
                 return nobar
             full_bar = Bar(
                 0,
-                max(1, ncols - _text_width(RE_ANSI.sub('', nobar)))
+                max(1, ncols - disp_len(nobar))
                 if ncols else 10,
                 charset=Bar.BLANK)
-            return bar_format.format(bar=full_bar, **format_dict)
+            res = bar_format.format(bar=full_bar, **format_dict)
+            return disp_trim(res, ncols) if ncols else res
         else:
             # no total: no progressbar, ETA, just progress stats
             return ((prefix + ": ") if prefix else '') + \
diff --git a/tqdm/utils.py b/tqdm/utils.py
--- a/tqdm/utils.py
+++ b/tqdm/utils.py
@@ -343,3 +343,27 @@ def _term_move_up():  # pragma: no cover
     def _text_width(s):
         return sum(
             2 if east_asian_width(ch) in 'FW' else 1 for ch in _unicode(s))
+
+
+def disp_len(data):
+    """
+    Returns the real on-screen length of a string which may contain
+    ANSI control codes and wide chars.
+    """
+    return _text_width(RE_ANSI.sub('', data))
+
+
+def disp_trim(data, length):
+    """
+    Trim a string which may contain ANSI control characters.
+    """
+    if len(data) == disp_len(data):
+        return data[:length]
+
+    ansi_present = bool(RE_ANSI.search(data))
+    while disp_len(data) > length:  # carefully delete one char at a time
+        data = data[:-1]
+    if ansi_present and bool(RE_ANSI.search(data)):
+        # assume ANSI reset is required
+        return data if data.endswith("\033[0m") else data + "\033[0m"
+    return data


Test patch:
diff --git a/tqdm/tests/tests_tqdm.py b/tqdm/tests/tests_tqdm.py
--- a/tqdm/tests/tests_tqdm.py
+++ b/tqdm/tests/tests_tqdm.py
@@ -9,6 +9,7 @@
 from nose import with_setup
 from nose.plugins.skip import SkipTest
 from nose.tools import assert_raises
+from nose.tools import eq_
 from contextlib import contextmanager
 from warnings import catch_warnings, simplefilter
 
@@ -273,6 +274,59 @@ def test_format_meter():
         "100kiB [00:13, 7.69kiB/s]"
     assert format_meter(100, 1000, 12, ncols=0, rate=7.33) == \
         " 10% 100/1000 [00:12<02:02,  7.33it/s]"
+    eq_(
+        # ncols is small, l_bar is too large
+        # l_bar gets chopped
+        # no bar
+        # no r_bar
+        format_meter(
+            0, 1000, 13, ncols=10,
+            bar_format="************{bar:10}$$$$$$$$$$"),
+        "**********"  # 10/12 stars since ncols is 10
+    )
+    eq_(
+        # n_cols allows for l_bar and some of bar
+        # l_bar displays
+        # bar gets chopped
+        # no r_bar
+        format_meter(
+            0, 1000, 13, ncols=20,
+            bar_format="************{bar:10}$$$$$$$$$$"),
+        "************        "  # all 12 stars and 8/10 bar parts
+    )
+    eq_(
+        # n_cols allows for l_bar, bar, and some of r_bar
+        # l_bar displays
+        # bar displays
+        # r_bar gets chopped
+        format_meter(
+            0, 1000, 13, ncols=30,
+            bar_format="************{bar:10}$$$$$$$$$$"),
+        "************          $$$$$$$$"
+        # all 12 stars and 10 bar parts, but only 8/10 dollar signs
+    )
+    eq_(
+        # trim left ANSI; escape is before trim zone
+        format_meter(
+            0, 1000, 13, ncols=10,
+            bar_format="*****\033[22m****\033[0m***{bar:10}$$$$$$$$$$"),
+        "*****\033[22m****\033[0m*\033[0m"
+        # we only know it has ANSI codes, so we append an END code anyway
+    )
+    eq_(
+        # trim left ANSI; escape is at trim zone
+        format_meter(
+            0, 1000, 13, ncols=10,
+            bar_format="*****\033[22m*****\033[0m**{bar:10}$$$$$$$$$$"),
+        "*****\033[22m*****\033[0m"
+    )
+    eq_(
+        # trim left ANSI; escape is after trim zone
+        format_meter(
+            0, 1000, 13, ncols=10,
+            bar_format="*****\033[22m******\033[0m*{bar:10}$$$$$$$$$$"),
+        "*****\033[22m*****\033[0m"
+    )
     # Check that bar_format correctly adapts {bar} size to the rest
     assert format_meter(20, 100, 12, ncols=13, rate=8.1,
                         bar_format=r'{l_bar}{bar}|{n_fmt}/{total_fmt}') == \
@@ -300,14 +354,18 @@ def test_format_meter():
 
 def test_ansi_escape_codes():
     """Test stripping of ANSI escape codes"""
-    format_meter = tqdm.format_meter
-    ansi = {'BOLD': '\033[1m',
-            'RED': '\033[91m',
-            'END': '\033[0m'}
-    desc = '{BOLD}{RED}Colored{END} description'.format(**ansi)
+    ansi = dict(BOLD='\033[1m', RED='\033[91m', END='\033[0m')
+    desc_raw = '{BOLD}{RED}Colored{END} description'
     ncols = 123
-    ansi_len = sum([len(code) for code in ansi.values()])
-    meter = format_meter(0, 100, 0, ncols=ncols, prefix=desc)
+
+    desc_stripped = desc_raw.format(BOLD='', RED='', END='')
+    meter = tqdm.format_meter(0, 100, 0, ncols=ncols, prefix=desc_stripped)
+    assert len(meter) == ncols
+
+    desc = desc_raw.format(**ansi)
+    meter = tqdm.format_meter(0, 100, 0, ncols=ncols, prefix=desc)
+    # `format_meter` inserts an extra END for safety
+    ansi_len = len(desc) - len(desc_stripped) + len(ansi['END'])
     assert len(meter) == ncols + ansi_len
 
 
@@ -901,7 +959,7 @@ def test_ascii():
     for ascii in [" .oO0", " #"]:
         with closing(StringIO()) as our_file:
             for _ in tqdm(_range(len(ascii) - 1), file=our_file, miniters=1,
-                          mininterval=0, ascii=ascii, ncols=1):
+                          mininterval=0, ascii=ascii, ncols=27):
                 pass
             res = our_file.getvalue().strip("\r").split("\r")
         for bar, line in zip(ascii, res):