Package: p-tqdm
p_tqdmp_tqdmmakes parallel processing with progress bars easy.p_tqdmis a wrapper aroundpathos.multiprocessingandtqdm. Unlike Python's default multiprocessing library, pathos provides a more flexible parallel map which can apply almost any type of function, including lambda functions, nested functions, and class methods, and can easily handle functions with multiple arguments. tqdm is applied on top of pathos's parallel map and displays a progress bar including an estimated time to completion.Installationpip install p_tqdmExampleLet's say you want to add two lists element by element. Without any parallelism, this can be done easily with a Pythonmap.l1=['1','2','3']l2=['a','b','c']defadd(a,b):returna+badded=map(add,l1,l2)# added == ['1a', '2b', '3c']But if the lists are much larger or the computation is more intense, parallelism becomes a necessity. However, the syntax is often cumbersome.p_tqdmmakes it easy and adds a progress bar too.fromp_tqdmimportp_mapadded=p_map(add,l1,l2)# added == ['1a', '2b', '3c']0%|                                    | 0/3 [00:00<?, ?it/s]
 33%|████████████                        | 1/3 [00:01<00:02, 1.00s/it]
 66%|████████████████████████            | 2/3 [00:02<00:01, 1.00s/it]
100%|████████████████████████████████████| 3/3 [00:03<00:00, 1.00s/it]p_tqdm functionsParallel mapsp_map- parallel ordered mapp_imap- iterator for parallel ordered mapp_umap- parallel unordered mapp_uimap- iterator for parallel unordered mapSequential mapst_map- sequential ordered mapt_imap- iterator for sequential ordered mapp_mapPerforms an ordered map in parallel.fromp_tqdmimportp_mapdefadd(a,b):returna+badded=p_map(add,['1','2','3'],['a','b','c'])# added = ['1a', '2b', '3c']p_imapReturns an iterator for an ordered map in parallel.fromp_tqdmimportp_imapdefadd(a,b):returna+biterator=p_imap(add,['1','2','3'],['a','b','c'])forresultiniterator:print(result)# prints '1a', '2b', '3c'p_umapPerforms an unordered map in parallel.fromp_tqdmimportp_umapdefadd(a,b):returna+badded=p_umap(add,['1','2','3'],['a','b','c'])# added is an array with '1a', '2b', '3c' in any orderp_uimapReturns an iterator for an unordered map in parallel.fromp_tqdmimportp_uimapdefadd(a,b):returna+biterator=p_uimap(add,['1','2','3'],['a','b','c'])forresultiniterator:print(result)# prints '1a', '2b', '3c' in any ordert_mapPerforms an ordered map sequentially.fromp_tqdmimportt_mapdefadd(a,b):returna+badded=t_map(add,['1','2','3'],['a','b','c'])# added == ['1a', '2b', '3c']t_imapReturns an iterator for an ordered map to be performed sequentially.fromp_tqdmimportp_imapdefadd(a,b):returna+biterator=t_imap(add,['1','2','3'],['a','b','c'])forresultiniterator:print(result)# prints '1a', '2b', '3c'Shared propertiesArgumentsAllp_tqdmfunctions accept any number of iterables as input, as long as the number of iterables matches the number of arguments of the function.To repeat a non-iterable argument along with the iterables, use Python'spartialfrom thefunctoolslibrary. See the example below.fromfunctoolsimportpartiall1=['1','2','3']l2=['a','b','c']defadd(a,b,c=''):returna+b+cadded=p_map(partial(add,c='!'),l1,l2)# added == ['1a!', '2b!', '3c!']CPUsAll the parallelp_tqdmfunctions can be passed the keywordnum_cpusto indicate how many CPUs to use. The default is all CPUs.num_cpuscan either be an integer to indicate the exact number of CPUs to use or a float to indicate the proportion of CPUs to use.Note that the parallel Pool objects used byp_tqdmare automatically closed when the map finishes processing.tqdm instanceAll the parallelp_tqdmfunctions can be passed the keywordtqdmto choose a specific flavor of tqdm. By default, this value is taken fromtqdm.auto. Thetqdmparameter can be used passp_tqdmoutput totqdm.gui,tqdm.tkor any customized subclass oftqdm.