Package: progress-parallel
the latest version of 3.2.1🎉Changes in the new version of3.2.1- Some serious bugs were fixed. -CAUTIONSerious bugs are fixed as hotfixes in v3.1.1.We have discovered a serious flaw in the functionality available in for-loop added in v3.0.1.This has been corrected in v3.1.1.Accordingly, v3.0.1 has been removed from PyPI.prplprpl(progress-parallel) is a library to visualize the progress of parallel processing byconcurrent.futures, the standard python library.Descriptionprplis a "Tips" library that makes the standard python parallel processing library simpler to use.The general functionality is the same asconcurrent.futuresitself, but it is possible to visualize the parallel processing status of threads generated using this library.More about prplInprpl, when performing calculations on numbers managed by a list, the list is automatically parallelized and parallelized.
The progress of the parallel processing can be visualized and the parallel processing can be checked.Standard features include the following,Display of thread progressres=prpl(target_list=t_list,target_function=test_calc_func)Display of progress barres=prpl(target_list=t_list,target_function=test_calc_func,symbol="#",smpl=True)Measure run timerres=prpl(target_list=t_list,target_function=test_calc_func,timer=True)Change the color of the progress bar.res=prpl(target_list=t_list,target_function=test_calc_func,symbol="#",color="green")Only single progress bar is available.res=prpl(target_list=t_list,target_function=test_calc_func,list_sep=1)For use with the for-loop.for_inprpl(t_list,symbol="#"):passWhen used in a for-loop, the color of the symbol can also be changed.However, this feature does not allow for graphical functions with arrows, such as multi-threading (parallel processing) mode. If the symbol argument is not used, a standard count-up progress bar is used.for_inprpl(t_list):passWhen multiple arguments are passed to a function.""" For functions with multiple arguments. """defmulti_arguments_target_function(a,b,c,d):# The argument that receives the elements of the list is "a".returna+b+c+dargs_dict={"b":2,"c":3,"d":4}res=prpl(target_list=t_list,target_function=multi_arguments_target_function,args=args_dict)When passing multiple arguments to a function, be sure to pass them as type ofdict.
Also, it is not necessary to include the target list in the arguments grouped together in that type ofdict.For other samples, test code is available attests directory. Please run it.(and gifs are atsample git directory)parallel processingFor parallel processing, it is CPU parallel processing using the standard python library.
For more information, check the pythonconcurrent.futuresdocumentation.Sample codeAn actual code sample example would be as follows.
An example of implementation in actual code is shown below.When performing some calculation process on a set of numbers compiled in a list, the conventional implementation is as follows:""" conventional method """calculation_result=[]target_list=list(range(0,100000))fortarget_varintarget_list:calc_answer=(target_var**target_var)**2# formulacalculation_result.append(calc_answer)By using prpl, the process inside a for loop statement is automatically executed in parallel by passing it as a function, and the result is returned.""" Separate formulas to be processed in the for loop as functions """deftarget_function(target_var):calc_answer=(target_var**target_var)**2# target formulareturncalc_answer""" Methods using prpl """fromprplimportprpltarget_list=list(range(0,100000))calculation_result=prpl(target_list,target_function)# prplThe actual operating gif of the comparison is as follows:It is important to note that if the formula is not complex, it is not suitable for parallel processing.
Because of the nature of parallel processing, prpl should not be used unnecessarily.Optional arguments, etcThe list of arguments, etc. that can be used inprplis as follows.result_list = prpl(  # The results will always return with a type of List.
   
   target_list,      # Required argument
      ### List to be used in the target function.

   target_function,  # May be required argument.
      ### A function that contains a target formula, etc, and must include a return statement.
      ### Required arguments if prpl is used in parallel processing visualization.
      ### It is not required if it is used in the for-loop.

   args,
      ### Multiple arguments managed by dictionary type.
   
   list_sep,
      ### Number of list divisions. 
      ### Default setting is 5.

   checkpoint,
      ### Number of breaks to display progress.
      ### ### Default setting is 10.
   
   title,
      ### Title of each progress bar.
      ### By default, the name of the target function.
   
   symbol,
      ### Progress bar mark settings.
   
   symbol_c,
      ### Setting the length of the progress bar.
      ### Default setting is 50.
   
   smpl,
      ### Simple display mode.
      ### Default setting is False.
   
   timer,
      ### Measuring run time.
      ### Not available in simple mode.

   color
      ### Change the color of the progress bar.

)Impossible and warning with prplTheenumerate()is not available.Normally,enumerate()is available in a for loop statement, but this is not available in a ptpl using concurrent.futures.Aprint()cannot be used within a function for which parallel processing is desired.Strictly speaking, you can use it. However, using aprint()is the same as executing a print statement inside a for loop, so the progress bar will be misaligned.Since parallel processing conforms to theconcurrent.futureslibrary, anything that cannot be done with concurrent.futures is not possible withprpl.Concurrent.futuresis used in prpl parallel processing. Therefore, what is impossible withconcurrent.futuresis also impossible withprpl. To the last,prplis a library to handle parallel processing in a simple and intuitive way, and a library to visualize the parallel processing.The display may shift on the console.During execution, the vertical display of the console is controlled in the program, but the horizontal length is not. Therefore, if theprpldisplay exceeds the length of the console's width, theprplmay be significantly disturbed.Be sure to provide a target function with return.By the nature of the program, it performs an arithmetic operation on the given "list-type" array and returns the result in a list type. Therefore,prplwill cause an error if some return is not made in the function.The following are appropriate and inappropriate examples.""" appropriate example target function """defOK_func(i):ans=(i**i)**2returnans# return""" inappropriate example """defNG_func(i):ans=(i**i)**2# not returnBe sure to include a return statement in the function.Getting StartedInstallingLatest prpl viaPyPI(pip install)pip install progress-parallelInstall by pip from githubpip install git+https://github.com/Domzou-kun/prpl.gitor install via SSHpip install git+git://github.com:Domzou-kun/prpl.gitParticularly technical notesdisplayprpl's, the progressbar on the console uses ESC and CSI sequences to control multiple lines simultaneously.When using CSI and ESC sequences with cmd, etc., terminal settings must be enabled.However, many of the programs that display such a progressbar as this one have modified kernel and console settings to suit their individual programs. This means that if other console-controlling programs are used at the same time, the settings may conflict and break the drawing.We have already confirmed that whentqdmandprplare imported and used at the same time, there is a conflict withtqdm's settings andprpl's display is corrupted.We are currently working on a fix for this problem.If you have a suggestion for a technical solution, please write to the issue. We will gladly review and consider it.AuthorsDomzoulinkThe link to PyPI is here.PyPI project linkこのreadmeを日本語にしたものになります。本readmeの重要な部分をメインに日本語に直してQiitaに投稿しています。Pythonでmultiprocessingを簡単に可視化してくれるライブラリを作ってみたVersion historyIf you want to know about past versions, please refer toversion history.LICENSEPyTorch has a MIT license, as found in theLICENSE file.