Package: atpbar
atpbarProgress barsfor threading and multiprocessing tasks on terminal
and Jupyter Notebook.100.00% :::::::::::::::::::::::::::::::::::::::: |     7811 /     7811 |:  task 1
 100.00% :::::::::::::::::::::::::::::::::::::::: |    23258 /    23258 |:  task 0
  65.62% ::::::::::::::::::::::::::               |     8018 /    12219 |:  task 4
  60.89% ::::::::::::::::::::::::                 |    31083 /    51048 |:  task 2
  25.03% ::::::::::                               |    23884 /    95421 |:  task 3atpbarcan display multiple progress bars simultaneously growing to
show the progresses of iterations of loops inthreadingormultiprocessingtasks.atpbarcan display progress bars on terminal andJupyter
Notebook.atpbarcan be used withMantichora.atpbarstarted its development in 2015 as part ofalphatwirl.atpbarprevented physicists from terminating their running analysis codes,
which would take many hours to complete, by showing progress bars
indicating their codes were actually running. The progress bars have
saved the physicists countless hours total.atpbarhad been the
sub-packageprogressbarof alphatwirl until it became an independent package, with the nameatpbar, in February 2019.You can try it on Jupyter Notebook online:RequirementInstallUser guideQuick startImport librariesOne loopNested loopsThreadingMultiprocessingWith MantichoraFeaturesAbreakand an exceptionProgress of starting threads and processes with progress barsOn Jupyter NotebookNon TTY deviceHow to disable progress barsLicenseContactRequirementPython 3.6, 3.7, or 3.8InstallYou can install withcondafrom conda-forge:condainstall-cconda-forgeatpbaror withpip:pipinstall-UatpbarUser guideQuick startI will show here how to use atpbar by simple examples.Import librariesTo create simple loops in the examples, we use two python standard
libraries,timeandrandom. Import the
two packages as well asatpbar.importtime,randomfromatpbarimportatpbarNote: import the objectatpbarfrom the packageatpbarrather
than importing the packageatpbaritself.One loopThe objectatpbaris an iterable that can wrap another iterable and
shows the progress bars for the iterations. (The idea of making the
interface iterable was inspired bytqdm.)n=random.randint(1000,10000)foriinatpbar(range(n)):time.sleep(0.0001)The task in the above code is to sleep for0.0001seconds in each
iteration of the loop. The number of the iterations of the loop is
randomly selected from between1000and10000.A progress bar will be shown byatpbar.51.25% ::::::::::::::::::::                     |     4132 /     8062 |:  range(0, 8062)In order foratpbarto show a progress bar, the wrapped iterable
needs to have a length. If the length cannot be obtained bylen(),atpbarwon't show a progress bar.Nested loopsatpbarcan show progress bars for nested loops as in the following
example.foriinatpbar(range(4),name='outer'):n=random.randint(1000,10000)forjinatpbar(range(n),name='inner{}'.format(i)):time.sleep(0.0001)The outer loop iterates 4 times. The inner loop is similar to the loop
in the previous example---sleeps for0.0001seconds. You can
optionally give the keyword argumentnameto specify the label on
the progress bar.100.00% :::::::::::::::::::::::::::::::::::::::: |     3287 /     3287 |:  inner 0
 100.00% :::::::::::::::::::::::::::::::::::::::: |     5850 /     5850 |:  inner 1
  50.00% ::::::::::::::::::::                     |        2 /        4 |:  outer  
  34.42% :::::::::::::                            |     1559 /     4529 |:  inner 2In the snapshot of the progress bars above, the outer loop is in its
3rd iteration. The inner loop has completed twice and is running the
third. The progress bars for the completed tasks move up. The progress
bars for the active tasks are growing at the bottom.Threadingatpbarcan show multiple progress bars for loops concurrently
iterating in different threads.The functionrun_with_threading()in the following code shows an
example.fromatpbarimportflushimportthreadingdefrun_with_threading():nthreads=5deftask(n,name):foriinatpbar(range(n),name=name):time.sleep(0.0001)threads=[]foriinrange(nthreads):name='thread{}'.format(i)n=random.randint(5,100000)t=threading.Thread(target=task,args=(n,name))t.start()threads.append(t)fortinthreads:t.join()flush()run_with_threading()The task to sleep for0.0001seconds is defined as the functiontask. Thetaskis concurrently run 5 times withthreading.atpbarcan be used in any threads. Five progress bars growing
simultaneously will be shown. The functionflush()returns when the
progress bars have finished updating.100.00% :::::::::::::::::::::::::::::::::::::::: |     8042 /     8042 |:  thread 3 
  33.30% :::::::::::::                            |    31967 /    95983 |:  thread 0 
  77.41% ::::::::::::::::::::::::::::::           |    32057 /    41411 |:  thread 1 
  45.78% ::::::::::::::::::                       |    31816 /    69499 |:  thread 2 
  39.93% :::::::::::::::                          |    32373 /    81077 |:  thread 4As a task completes, the progress bar for the task moves up. The
progress bars for active tasks are at the bottom.Multiprocessingatpbarcan be used withmultiprocessing.The functionrun_with_multiprocessing()in the following code shows
an example.importmultiprocessingmultiprocessing.set_start_method('fork',force=True)fromatpbarimportregister_reporter,find_reporter,flushdefrun_with_multiprocessing():deftask(n,name):foriinatpbar(range(n),name=name):time.sleep(0.0001)defworker(reporter,task,queue):register_reporter(reporter)whileTrue:args=queue.get()ifargsisNone:queue.task_done()breaktask(*args)queue.task_done()nprocesses=4ntasks=10reporter=find_reporter()queue=multiprocessing.JoinableQueue()foriinrange(nprocesses):p=multiprocessing.Process(target=worker,args=(reporter,task,queue))p.start()foriinrange(ntasks):name='task{}'.format(i)n=random.randint(5,100000)queue.put((n,name))foriinrange(nprocesses):queue.put(None)queue.join()flush()run_with_multiprocessing()It starts four workers in subprocesses withmultiprocessingand have
them run ten tasks.In order to useatpbarin a subprocess, thereporter, which can be
found in the main process by the functionfind_reporter(), needs to
be brought to the subprocess and registered there by the functionregister_reporter().Simultaneously growing progress bars will be shown.100.00% :::::::::::::::::::::::::::::::::::::::: |    44714 /    44714 |:  task 3
 100.00% :::::::::::::::::::::::::::::::::::::::: |    47951 /    47951 |:  task 2
 100.00% :::::::::::::::::::::::::::::::::::::::: |    21461 /    21461 |:  task 5
 100.00% :::::::::::::::::::::::::::::::::::::::: |    73721 /    73721 |:  task 1
 100.00% :::::::::::::::::::::::::::::::::::::::: |    31976 /    31976 |:  task 4
 100.00% :::::::::::::::::::::::::::::::::::::::: |    80765 /    80765 |:  task 0
  58.12% :::::::::::::::::::::::                  |    20133 /    34641 |:  task 6
  20.47% ::::::::                                 |    16194 /    79126 |:  task 7
  47.71% :::::::::::::::::::                      |    13072 /    27397 |:  task 8
  76.09% ::::::::::::::::::::::::::::::           |     9266 /    12177 |:  task 9With MantichoraMantichoraprovides a
simple interface tomultiprocessing.With Mantichora,task()can be concurrently run with multiprocessing
with as simple code as the following example:frommantichoraimportmantichoradeftask(name):n=random.randint(1000,10000)foriinatpbar(range(n),name=name):time.sleep(0.0001)withmantichora()asmcore:mcore.run(task,'task 1')mcore.run(task,'task 2')mcore.run(task,'task 3')mcore.run(task,'task 4')mcore.run(task,'task 5')returns=mcore.returns()atpbarcan be used in the task function.100.00% :::::::::::::::::::::::::::::::::::::::: |     2288 /     2288 |:  task 3
 100.00% :::::::::::::::::::::::::::::::::::::::: |     3964 /     3964 |:  task 4
 100.00% :::::::::::::::::::::::::::::::::::::::: |     4207 /     4207 |:  task 2
 100.00% :::::::::::::::::::::::::::::::::::::::: |     8519 /     8519 |:  task 1
 100.00% :::::::::::::::::::::::::::::::::::::::: |     6595 /     6595 |:  task 5FeaturesAbreakand an exceptionWhen the loop ends with abreakor an exception, the progress bar stops with
the last complete iteration.For example, the loop in the following code breaks during the 1235th iteration.foriinatpbar(range(2000)):ifi==1234:breaktime.sleep(0.0001)Sinceistarts with0, wheniis1234, the loop is in its 1235th
iteration. The last complete iteration is 1234. The progress bar stops at 1234.61.70% ::::::::::::::::::::::::                 |     1234 /     2000 |:  range(0, 2000)As an example of an exception, in the following code, an exception is
thrown during the 1235th iteration.foriinatpbar(range(2000)):ifi==1234:raiseExceptiontime.sleep(0.0001)The progress bar stops at the last complete iteration, 1234.61.70% ::::::::::::::::::::::::                 |     1234 /     2000 |:  range(0, 2000)
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ExceptionThis feature works as well with nested loops, threading, and
multiprocessing. For example, in the following code, the loops in five
threads break at 1235th iteration.fromatpbarimportflushimportthreadingdefrun_with_threading():deftask(n,name):foriinatpbar(range(n),name=name):ifi==1234:breaktime.sleep(0.0001)nthreads=5threads=[]foriinrange(nthreads):name='thread{}'.format(i)n=random.randint(3000,10000)t=threading.Thread(target=task,args=(n,name))t.start()threads.append(t)fortinthreads:t.join()flush()run_with_threading()All progress bars stop at 1234.18.21% :::::::                                  |     1234 /     6777 |:  thread 0
  15.08% ::::::                                   |     1234 /     8183 |:  thread 2
  15.25% ::::::                                   |     1234 /     8092 |:  thread 1
  39.90% :::::::::::::::                          |     1234 /     3093 |:  thread 4
  19.67% :::::::                                  |     1234 /     6274 |:  thread 3Progress of starting threads and processes with progress barsatpbarcan be used for a loop that starts sub-threads or
sub-processes in whichatpbaris also used.fromatpbarimportflushimportthreadingdefrun_with_threading():deftask(n,name):foriinatpbar(range(n),name=name):time.sleep(0.0001)nthreads=5threads=[]foriinatpbar(range(nthreads)):name='thread{}'.format(i)n=random.randint(200,1000)t=threading.Thread(target=task,args=(n,name))t.start()threads.append(t)time.sleep(0.1)fortinthreads:t.join()flush()run_with_threading()100.00% :::::::::::::::::::::::::::::::::::::::: |      209 /      209 |:  thread 1
 100.00% :::::::::::::::::::::::::::::::::::::::: |      699 /      699 |:  thread 0
 100.00% :::::::::::::::::::::::::::::::::::::::: |      775 /      775 |:  thread 2
 100.00% :::::::::::::::::::::::::::::::::::::::: |      495 /      495 |:  thread 3
 100.00% :::::::::::::::::::::::::::::::::::::::: |        5 /        5 |:  range(0, 5)
 100.00% :::::::::::::::::::::::::::::::::::::::: |      647 /      647 |:  thread 4Theatpbarsensibly works regardless of the order in which multiple
instances ofatpbarin multiple threads and multiple processes start
and end. The progress bars in the example above indicates that the
loops in four threads have already ended before the loop in the main
threads ended; the loop in the last thread ended afterwards.On Jupyter NotebookOn Jupyter Notebook,atpbarshows progress bars based onwidgets.You can try interactively online:Non TTY deviceIf neither on Jupyter Notebook or on a TTY device,atpbaris not
able to show progress bars.atpbaroccasionally prints the status.03/04 09:17 :     1173 /     7685 ( 15.26%): thread 0 
03/04 09:17 :     1173 /     6470 ( 18.13%): thread 3 
03/04 09:17 :     1199 /     1199 (100.00%): thread 4 
03/04 09:18 :     1756 /     2629 ( 66.79%): thread 2 
03/04 09:18 :     1757 /     7685 ( 22.86%): thread 0 
03/04 09:18 :     1757 /     6470 ( 27.16%): thread 3 
03/04 09:19 :     2342 /     2629 ( 89.08%): thread 2How to disable progress barsThe functiondisable()disablesatpbar; progress bars will not be shown.fromatpbarimportdisabledisable()This function needs to be called beforeatpbarorfind_reporter()is used,
typically at the beginning of the program.Licenseatpbaris licensed under the BSD license.ContactTai Sakuma -tai.sakuma@gmail.com