Repository: tqdm/tqdm (Python)

Issue:
The canonical multiprocessing example displays only a single bar
I'm trying to use `tqdm` to monitor multiple FTP downloads using `ftplib`. Browsing around the documentation led me to this "canonical" example:

```
from time import sleep
from tqdm import tqdm
from multiprocessing import Pool, freeze_support

def progresser(n):
    text = "progresser #{}".format(n)
    for i in tqdm(range(5000), desc=text, position=n):
        sleep(0.001)

if __name__ == '__main__':
    freeze_support()  # for Windows support
    L = list(range(10))
    Pool(len(L)).map(progresser, L)
```

but on my system (Ubuntu 14.04, Python 3.4.3, tqdm 4.14.0) I get a mixture of empty lines, non-updating progress bars, and a single updating progress bar at the bottom:

![selection_035](https://user-images.githubusercontent.com/893512/27547700-b3b3687c-5a8e-11e7-8714-4d70933afbc2.png)

If I watch the bottom progress bar carefully I can see that the name occasionally changes, so I think that the progress bars are being drawn on top of one another. I've searched the issues and the closest thing I can find is this:

https://github.com/tqdm/tqdm/issues/285

but I don't think it's the same problem, as that issue reported multiple updating progress bars visible and I am only seeing one (the screenshots are very different). Let me know if I can provide any other info - I really want to use `tqdm` for this project but am out of depth when it comes to trying to figure out what's wrong. 

MNT: set stack level on warnings
- [x] I have visited the [source website], and in particular
  read the [known issues]
- [x] I have searched through the [issue tracker] for duplicates
- [x] I have mentioned version numbers, operating system and
  environment, where applicable:
  ```python
  import tqdm, sys
  print(tqdm.__version__, sys.version, sys.platform)
  ```
- [x] If applicable, I have mentioned the relevant/related issue(s)

  [source website]: https://github.com/tqdm/tqdm/
  [known issues]: https://github.com/tqdm/tqdm/#faq-and-known-issues
  [issue tracker]: https://github.com/tqdm/tqdm/issues?q=

This makes it much easier to find where in client code needs to be
fixed.

Adds stacklevel kwarg to warnings added in 4.36.0


Hints:
This is annoying. It works fine on py27... There is indeed something weird with py3. Have you seen #329?
I did have a look at #329, but quickly reached the limit of my understanding when looking at the code examples :-). Entertainingly, when I run the canonical code under Python 2 on my system, I get a different result, though still not quite what is expected:

![selection_036](https://user-images.githubusercontent.com/893512/27558319-f1355e0a-5ab4-11e7-9034-ac7443c4d5c8.png)

Which seems to me like the progress bars are still overwriting each other, just in a different pattern. 
The initialization of tdqm, update method and close method are not thread-safe. I searchd the issue, and found that pull#291 is trying to solve the parallelism output, but that code is not in current release, nor in master branch. You can use your own synchronization mechanism to solve this problem.

Besides, the progressbar at position 0 will have an extra line return appended when close it, as @lrq3000 mentioned in issue #329. An easy workaround is, starting from 1. I think it would be better to have an optinal parameter to determine this behavior.

A sample code 

```python
from multiprocessing.pool import ThreadPool
import time
import threading
from tqdm import tqdm


def demo(lock, position, total):
    text = "progresser #{}".format(position)
    with lock:
        progress = tqdm(
            total=total,
            position=position,
            desc=text,
        )
    for _ in range(0, total, 5):
        with lock:
            progress.update(5)
        time.sleep(0.1)
    with lock:
        progress.close()


pool = ThreadPool(5)
tasks = range(50)
lock = threading.Lock()
for i, url in enumerate(tasks, 1):
    pool.apply_async(demo, args=(lock, i, 100))
pool.close()
pool.join()
```

![screenshot](https://user-images.githubusercontent.com/1233483/29389305-5354c3f8-82b7-11e7-8946-a84e7e9085d3.png)


@NoAnyLove, I've been trying to sort this issue out for a couple of days. None of the stackexchange questions I found were helpful at all..this is PERFECT and exactly my use case. Thank you for posting this!!
Thanks for this. I recall writing pretty much exactly this snippet ages ago but it must be lost in another issue somewhere.
Ah yes of course good ol' #329
Could you check out the updated branch at #329, and verify it works with:

```python
from multiprocessing.pool import ThreadPool
import time
import threading
from tqdm import tqdm


def demo(position, total):
    text = "progresser #{}".format(position)
    progress = tqdm(
        total=total,
        position=position,
        desc=text,
    )
    for _ in range(0, total, 5):
        progress.update(5)
        time.sleep(0.1)
    progress.close()


pool = ThreadPool(5)
tasks = range(50)
for i, url in enumerate(tasks, 1):
    pool.apply_async(demo, args=(i, 100))
pool.close()
pool.join()
```

or even the original (canonical) example:

```python
from time import sleep
from tqdm import tqdm
from multiprocessing import Pool, freeze_support

def progresser(n):
    text = "progresser #{}".format(n)
    for i in tqdm(range(5000), desc=text, position=n):
        sleep(0.001)

if __name__ == '__main__':
    freeze_support()  # for Windows support
    L = list(range(10))
    Pool(len(L)).map(progresser, L)
```
It still does not work on my Mac. The result is as follows:

<img width="570" alt="image" src="https://user-images.githubusercontent.com/13083271/47271705-5e827a80-d541-11e8-8207-0c24e978febb.png">


multiprocessing library updates Pool to class, I used this, but there's still bug:

```
from multiprocessing import Pool, Lock
import time
from tqdm import tqdm
lock = Lock()

def demo(position):
    text = "progresser #{}".format(position)
    total = 100
    with lock:
        progress = tqdm(total=total, position=position, desc=text)
    for _ in range(0, total, 5):
        with lock:
            progress.update(5)
        time.sleep(0.1)
    with lock:
        progress.close()

args = []
for i in range(50):
    args.append((i,))

with Pool(processes=5) as pool:
    pool.starmap(demo, args)
```
![image](https://user-images.githubusercontent.com/10788062/57945449-eb0ed480-78a7-11e9-9999-67004757eec5.png)

```python
from fastprogress import progress_bar
from multiprocessing import Pool

def process(n: int) -> int:
    return n ** 2

numbers = [1, 2, 3, 4]

with Pool(3) as p:
    processed_numbers = list(tqdm(p.imap(process, numbers), total=len(numbers))
```
@pronkinnikita are you saying that if you replace `from fastprogress import progress_bar` with `from tqdm.auto import tqdm as progress_bar` your code stops working properly?
> @pronkinnikita are you saying that if you replace `from fastprogress import progress_bar` with `from tqdm.auto import tqdm as progress_bar` your code stops working properly?

Sorry, updated comment, now it seems fine. Nope, i think is works the same way with vanila tqdm.


Patch:
diff --git a/examples/parallel_bars.py b/examples/parallel_bars.py
--- a/examples/parallel_bars.py
+++ b/examples/parallel_bars.py
@@ -1,38 +1,41 @@
 from __future__ import print_function
 from time import sleep
 from tqdm import tqdm, trange
+from random import random
 from multiprocessing import Pool, freeze_support
 from concurrent.futures import ThreadPoolExecutor
 from functools import partial
 import sys
 
+NUM_SUBITERS = 9
+PY2 = sys.version_info[:1] <= (2,)
 
-L = list(range(9))[::-1]
 
-
-def progresser(n, auto_position=False):
-    interval = 0.001 / (len(L) - n + 2)
+def progresser(n, auto_position=True, write_safe=False):
+    interval = random() * 0.002 / (NUM_SUBITERS - n + 2)
     total = 5000
     text = "#{}, est. {:<04.2}s".format(n, interval * total)
     for _ in tqdm(range(total), desc=text, position=None if auto_position else n):
         sleep(interval)
     # NB: may not clear instances with higher `position` upon completion
     # since this worker may not know about other bars #796
-    if auto_position:
+    if write_safe:
         # we think we know about other bars (currently only py3 threading)
         if n == 6:
             tqdm.write("n == 6 completed")
 
-if sys.version_info[:1] > (2,):
-    progresser_thread = partial(progresser, auto_position=True)
-else:
-    progresser_thread = progresser
-
 
 if __name__ == '__main__':
     freeze_support()  # for Windows support
+    L = list(range(NUM_SUBITERS))[::-1]
+
+    print("Manual nesting")
+    for i in trange(16, desc="1"):
+        for _ in trange(16, desc="2 @ %d" % i, leave=i % 2):
+            sleep(0.01)
+
     print("Multi-processing")
-    p = Pool(len(L), initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
+    p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
     p.map(progresser, L)
 
     # unfortunately need ncols
@@ -41,10 +44,6 @@ def progresser(n, auto_position=False):
         ncols = t.ncols or 80
     print(("{msg:<{ncols}}").format(msg="Multi-threading", ncols=ncols))
 
-    with ThreadPoolExecutor(4) as p:
+    with ThreadPoolExecutor() as p:
+        progresser_thread = partial(progresser, write_safe=not PY2)
         p.map(progresser_thread, L)
-
-    print("Manual nesting")
-    for i in trange(16, desc="1"):
-        for _ in trange(16, desc="2 @ %d" % i, leave=i % 2):
-            sleep(0.01)
diff --git a/examples/tqdm_wget.py b/examples/tqdm_wget.py
--- a/examples/tqdm_wget.py
+++ b/examples/tqdm_wget.py
@@ -48,9 +48,10 @@ def update_to(b=1, bsize=1, tsize=None):
         bsize  : int, optional
             Size of each block (in tqdm units) [default: 1].
         tsize  : int, optional
-            Total size (in tqdm units). If [default: None] remains unchanged.
+            Total size (in tqdm units). If [default: None] or -1,
+            remains unchanged.
         """
-        if tsize is not None:
+        if tsize not in (None, -1):
             t.total = tsize
         t.update((b - last_b[0]) * bsize)
         last_b[0] = b
diff --git a/tqdm/__init__.py b/tqdm/__init__.py
--- a/tqdm/__init__.py
+++ b/tqdm/__init__.py
@@ -24,7 +24,7 @@ def tqdm_notebook(*args, **kwargs):  # pragma: no cover
     from warnings import warn
     warn("This function will be removed in tqdm==5.0.0\n"
          "Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`",
-         TqdmDeprecationWarning)
+         TqdmDeprecationWarning, stacklevel=2)
     return _tqdm_notebook(*args, **kwargs)
 
 
@@ -36,5 +36,5 @@ def tnrange(*args, **kwargs):  # pragma: no cover
     from .notebook import trange as _tnrange
     from warnings import warn
     warn("Please use `tqdm.notebook.trange` instead of `tqdm.tnrange`",
-         TqdmDeprecationWarning)
+         TqdmDeprecationWarning, stacklevel=2)
     return _tnrange(*args, **kwargs)
diff --git a/tqdm/_main.py b/tqdm/_main.py
--- a/tqdm/_main.py
+++ b/tqdm/_main.py
@@ -4,4 +4,4 @@
 from warnings import warn
 warn("This function will be removed in tqdm==5.0.0\n"
      "Please use `tqdm.cli.*` instead of `tqdm._main.*`",
-     TqdmDeprecationWarning)
+     TqdmDeprecationWarning, stacklevel=2)
diff --git a/tqdm/_monitor.py b/tqdm/_monitor.py
--- a/tqdm/_monitor.py
+++ b/tqdm/_monitor.py
@@ -93,7 +93,7 @@ def run(self):
                 if instances != self.get_instances():  # pragma: nocover
                     warn("Set changed size during iteration" +
                          " (see https://github.com/tqdm/tqdm/issues/481)",
-                         TqdmSynchronisationWarning)
+                         TqdmSynchronisationWarning, stacklevel=2)
 
     def report(self):
         return not self.was_killed.is_set()
diff --git a/tqdm/_tqdm.py b/tqdm/_tqdm.py
--- a/tqdm/_tqdm.py
+++ b/tqdm/_tqdm.py
@@ -4,4 +4,4 @@
 from warnings import warn
 warn("This function will be removed in tqdm==5.0.0\n"
      "Please use `tqdm.std.*` instead of `tqdm._tqdm.*`",
-     TqdmDeprecationWarning)
+     TqdmDeprecationWarning, stacklevel=2)
diff --git a/tqdm/_tqdm_gui.py b/tqdm/_tqdm_gui.py
--- a/tqdm/_tqdm_gui.py
+++ b/tqdm/_tqdm_gui.py
@@ -4,4 +4,4 @@
 from warnings import warn
 warn("This function will be removed in tqdm==5.0.0\n"
      "Please use `tqdm.gui.*` instead of `tqdm._tqdm_gui.*`",
-     TqdmDeprecationWarning)
+     TqdmDeprecationWarning, stacklevel=2)
diff --git a/tqdm/_tqdm_notebook.py b/tqdm/_tqdm_notebook.py
--- a/tqdm/_tqdm_notebook.py
+++ b/tqdm/_tqdm_notebook.py
@@ -4,4 +4,4 @@
 from warnings import warn
 warn("This function will be removed in tqdm==5.0.0\n"
      "Please use `tqdm.notebook.*` instead of `tqdm._tqdm_notebook.*`",
-     TqdmDeprecationWarning)
+     TqdmDeprecationWarning, stacklevel=2)
diff --git a/tqdm/_utils.py b/tqdm/_utils.py
--- a/tqdm/_utils.py
+++ b/tqdm/_utils.py
@@ -3,4 +3,4 @@
 from warnings import warn
 warn("This function will be removed in tqdm==5.0.0\n"
      "Please use `tqdm.utils.*` instead of `tqdm._utils.*`",
-     TqdmDeprecationWarning)
+     TqdmDeprecationWarning, stacklevel=2)
diff --git a/tqdm/autonotebook.py b/tqdm/autonotebook.py
--- a/tqdm/autonotebook.py
+++ b/tqdm/autonotebook.py
@@ -14,5 +14,5 @@
     from warnings import warn
     warn("Using `tqdm.autonotebook.tqdm` in notebook mode."
          " Use `tqdm.tqdm` instead to force console mode"
-         " (e.g. in jupyter console)", TqdmExperimentalWarning)
+         " (e.g. in jupyter console)", TqdmExperimentalWarning, stacklevel=2)
 __all__ = ["tqdm", "trange"]
diff --git a/tqdm/gui.py b/tqdm/gui.py
--- a/tqdm/gui.py
+++ b/tqdm/gui.py
@@ -41,7 +41,7 @@ def __init__(self, *args, **kwargs):
         if self.disable or not kwargs['gui']:
             return
 
-        warn('GUI is experimental/alpha', TqdmExperimentalWarning)
+        warn('GUI is experimental/alpha', TqdmExperimentalWarning, stacklevel=2)
         self.mpl = mpl
         self.plt = plt
         self.sp = None
diff --git a/tqdm/std.py b/tqdm/std.py
--- a/tqdm/std.py
+++ b/tqdm/std.py
@@ -12,7 +12,7 @@
 from __future__ import division
 # compatibility functions and utilities
 from .utils import _supports_unicode, _environ_cols_wrapper, _range, _unich, \
-    _term_move_up, _unicode, WeakSet, _basestring, _OrderedDict, \
+    _term_move_up, _unicode, WeakSet, _basestring, _OrderedDict, _text_width, \
     Comparable, RE_ANSI, _is_ascii, SimpleTextIOWrapper, FormatReplace
 from ._monitor import TMonitor
 # native libraries
@@ -474,7 +474,7 @@ def format_meter(n, total, elapsed, ncols=None, prefix='', ascii=False,
             # Formatting progress bar space available for bar's display
             full_bar = Bar(
                 frac,
-                max(1, ncols - len(RE_ANSI.sub('', nobar))) if ncols else 10,
+                max(1, ncols - _text_width(RE_ANSI.sub('', nobar))) if ncols else 10,
                 charset=Bar.ASCII if ascii is True else ascii or Bar.UTF)
             if not _is_ascii(full_bar.charset) and _is_ascii(bar_format):
                 bar_format = _unicode(bar_format)
@@ -490,7 +490,7 @@ def format_meter(n, total, elapsed, ncols=None, prefix='', ascii=False,
                 return nobar
             full_bar = Bar(
                 0,
-                max(1, ncols - len(RE_ANSI.sub('', nobar))) if ncols else 10,
+                max(1, ncols - _text_width(RE_ANSI.sub('', nobar))) if ncols else 10,
                 charset=Bar.BLANK)
             return bar_format.format(bar=full_bar, **format_dict)
         else:
@@ -516,7 +516,7 @@ def __new__(cls, *args, **kwargs):
                 except Exception as e:  # pragma: nocover
                     warn("tqdm:disabling monitor support"
                          " (monitor_interval = 0) due to:\n" + str(e),
-                         TqdmMonitorWarning)
+                         TqdmMonitorWarning, stacklevel=2)
                     cls.monitor_interval = 0
         # Return the instance
         return instance
@@ -645,7 +645,10 @@ def pandas(tclass, *targs, **tkwargs):
         """
         from pandas.core.frame import DataFrame
         from pandas.core.series import Series
-        from pandas import Panel
+        try:
+            from pandas import Panel
+        except ImportError:  # TODO: pandas>0.25.2
+            Panel = None
         try:  # pandas>=0.18.0
             from pandas.core.window import _Rolling_and_Expanding
         except ImportError:  # pragma: no cover
@@ -752,7 +755,8 @@ def wrapper(*args, **kwargs):
         DataFrameGroupBy.progress_apply = inner_generator()
         DataFrame.progress_applymap = inner_generator('applymap')
 
-        Panel.progress_apply = inner_generator()
+        if Panel is not None:
+            Panel.progress_apply = inner_generator()
         if PanelGroupBy is not None:
             PanelGroupBy.progress_apply = inner_generator()
 
@@ -1033,7 +1037,7 @@ def __exit__(self, exc_type, exc_value, traceback):
             # maybe eager thread cleanup upon external error
             if (exc_type, exc_value, traceback) == (None, None, None):
                 raise
-            warn("AttributeError ignored", TqdmWarning)
+            warn("AttributeError ignored", TqdmWarning, stacklevel=2)
 
     def __del__(self):
         self.close()
diff --git a/tqdm/utils.py b/tqdm/utils.py
--- a/tqdm/utils.py
+++ b/tqdm/utils.py
@@ -283,3 +283,13 @@ def _environ_cols_linux(fp):  # pragma: no cover
 
 def _term_move_up():  # pragma: no cover
     return '' if (os.name == 'nt') and (colorama is None) else '\x1b[A'
+
+
+try:
+    # TODO consider using wcswidth third-party package for 0-width characters
+    from unicodedata import east_asian_width
+except ImportError:
+    _text_width = len
+else:
+    def _text_width(s):
+        return sum(2 if east_asian_width(ch) in 'FW' else 1 for ch in _unicode(s))


Test patch:
diff --git a/tqdm/tests/tests_tqdm.py b/tqdm/tests/tests_tqdm.py
--- a/tqdm/tests/tests_tqdm.py
+++ b/tqdm/tests/tests_tqdm.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 # Advice: use repr(our_file.read()) to print the full output of tqdm
 # (else '\r' will replace the previous lines and you'll see only the latest.
 
@@ -277,6 +278,12 @@ def test_format_meter():
     assert format_meter(20, 100, 12, ncols=14, rate=8.1,
                         bar_format=r'{l_bar}{bar}|{n_fmt}/{total_fmt}') == \
         " 20%|" + unich(0x258d) + " |20/100"
+    # Check wide characters
+    if sys.version_info >= (3,):
+        assert format_meter(0, 1000, 13, ncols=68, prefix='ｆｕｌｌｗｉｄｔｈ: ') == \
+            "ｆｕｌｌｗｉｄｔｈ:   0%|                  | 0/1000 [00:13<?, ?it/s]"
+        assert format_meter(0, 1000, 13, ncols=68, prefix='ニッポン [ﾆｯﾎﾟﾝ]: ') == \
+            "ニッポン [ﾆｯﾎﾟﾝ]:   0%|                    | 0/1000 [00:13<?, ?it/s]"
     # Check that bar_format can print only {bar} or just one side
     assert format_meter(20, 100, 12, ncols=2, rate=8.1,
                         bar_format=r'{bar}') == \