Package: fast-map
List of contentsIntroductionCharacteristics of fast_map functionUsageInstallationPerformance comparison(against multithreading/multiprocessing on their own)Troubleshooting and issuesConsiderationsIntroductionWhat is a map?mapis a python function which allows to repetitively execute the same function without the need to use loops. It executes each task sequentially, meaning that it doesn't start executing a new task before completing the previous one.This library allows to execute multiple tasks in parallel using multiple processor cores, and multiple threads to maximise performance when the function is blocking (e.g. it's delayed bytime.sleep()).It providesfast_mapfunction and the non-blockingfast_map_asyncequivalent (having the same performance, but allowing to continue execution and receive results in callbacks).How is this useful?Let's take a look at the following function:deff(x):time.sleep(1)Usingmap(f, range(60))would take a minute to complete, whereasfast_map(f, range(60))would complete in around 1 second. Note that both of these functions return agenerator.Characteristics of fast_map functionprovides parallelism and concurrency for blocking functionsreturns agenerator(meaning that individual returned values are returned immediately after being computed, before the whole collection is returned as a whole)return is ordered (accordingly to supplied arguments), however the execution order of tasksisn't guaranteed* and will most likely differevenly distributes tasks within processesuses the number of threads equal to the number of supplied tasks (unless threads_limit argument is provided)uses the number of processes equal to the number of CPU cores unless the number of tasks (or suppliedthreads_limit/procs_limit) is smaller than it (e.g. to avoid creating multiple processes for a single task)threads_limitandprocs_limitarguments are optional (settingthreads_limitis strongly encouraged,procs_limitis only useful when want the number of created processes to be less than the number of CPU cores and the number of tasks/threads)*Regarding task execution orderTasks are passed to separate processes in their original order (attempting to produce ordered returns as fast as possible. However, tasks are executed in parallel, and there is no mechanism implemented in this library to ensure their start/end point will be ordered/synchronized, only their returned values are ordered.What does it mean?The code below will print numbers in the wrong order.deff(x):print(x)foriinfast_map(f,range(60)):passThe code below will print numbers in the correct order.deff(x):returnxforiinfast_map(f,range(60)):print(i)Usagefast_map (seefast_map_usage.pyfor a more elaborated demonstration.fromfast_mapimportfast_mapimporttimedefio_and_cpu_expensive_function(x):time.sleep(1)foriinrange(10**6):passreturnx*xforiinfast_map(io_and_cpu_expensive_function,range(8),threads_limit=100,procs_limit=10):print(i)Note that "threads_limit" has no effect here because only 8 threads are created anyway (1 for each task). It would make a difference if we used "range(101)". In such case we would have to wait additional second before the last (or few remaining) result was yielded/returned.Theprocs_limitonly takes effect if it's lower than the number of CPU cores and lower than the number of tasks to execute.fast_map_async (seefast_map_async_usage.pyfor a more elaborated demonstration)fromfast_mapimportfast_map_asyncimporttimedefio_and_cpu_expensive_function(x):time.sleep(1)foriinrange(10**6):passreturnx*xdefon_result(result):print(result)defon_done():print('all done')# returns a threadt=fast_map_async(io_and_cpu_expensive_function,range(8),on_result=on_result,on_done=on_donethreads_limit=100,procs_limit=4)t.join()Again, "threads_limit" has no effect here.Installationpython3 -m pip install fast_mapPerformance comparisonI compared it against using muliprocessing/multithreading on their own.test_fast_map.pyis the script I used. It was tested with:Python3.7Ubuntu 18.04.6Intel i5-3320M (4 cores)8GB DDR3 memoryResults show that for IO+CPU expensive tasks fast_map performs better than multithreading-only and multiprocessing-only approaches. For strictly CPU expensive tasks it performs better than multithreading-only but slightly worse than multiprocessing-only approach.In both cases, IO+CPU and strictly CPU expensive tasks, it performs better than the standard map.IO and CPU expensive taskStandard map is not shown because it would take minutes (as it executes tasks sequentially)."-1" result means that ProcessPoolExecutor failed due to "too many files open" (which on my system happens when around 1000 processes are created by the python script). It shows why creating large number of processes to achieve concurrency may be a bad idea. A better idea would be to either:rely on multi-threading itself (which unfortunately utilizes only a single cpu-core)use asyncio (assumming that the blocking code can be turned into coroutines), possibly combined with multiprocessing as shown inasyncioevalcombine multiprocessing with multi-threading just like fast_map doesThe following blocking function was used to produce the graph above:defio_and_cpu_expensive_blocking_function(x):time.sleep(1)foriinrange(10**6):passreturnxStrictly CPU expensive taskIt can be noticed that using larger number of threads tends to compute results faster even in CPU expensive tasks, however I would risk a statement that using such large number of threads (e.g. 1 per each task) for a stricly CPU expensive tasks may bring negligible speed improvement of the fast_map but may possibly slow down the whole system. Because python processes may "fight" with other process over CPU time (that's just my hypothesis).The following blocking function was used to produce the graph above:defcpu_expensive_blocking_function(x):foriinrange(10**6):passreturnxTroubleshooting and issuesIt isn't suitable to be used in multi-processing scripts unless you know what you're doing (it was problematic when I tried to use it in such scripts).Calling fast_map from different threads or calling fast_map_async in a loop may lead to creating too many processes or threads (usethreads_limitandprocs_limitarguments to avoid issues in such case).Accessing thread-safe objects (created externally, and using locks under the hood) within the function supplied to fast_map will probably result in a deadlock.By default the fast_mapthreads_limitparameter isNone, meaning that a separate thread is spawned foreachof supplied tasks (attempting to provide full concurrency). It is strongly encouraged to set threads_limit to some reasonable value for 2 reasons:large number of threads will slow down the CPU-expensive part of the blocking functionfast_map will result in unhandled exception when too many threads try to be created (on my system it's around few thousands)(btw if threads_limit is higher than the number of supplied tasks, then the number of created threads equals the number of supplied tasks, so threads_limit doesn't force the number of created threads, it only limits them)Implementation detailsfast_map uses multiprocessing module and its default process start method (which I believe isforkon Unix). It spawns the number of processes equal to the number of CPU cores. For each spawned process it uses a separate task supplyingmultiprocessing.Queue(each has its own for the sake of even task distribution). It uses a singl common results queue for collecting results. It usesconcurrent.futures.ThreadPoolExecutorto implement multi-threading. It uses a singlethreading.Threadto enqueue all the tasks (this allows to start computation on multiple processes without the need to enqueue all the tasks first).It was inspired by a similar project which combined multiprocessing with asyncio:asyncioevalby Nicholas BaskerConsiderationsWhy not use threading or multiprocessing on their own?Multithreading in Python uses a single core on multi-core processors. Multiprocessing isn't well suited to provide concurrency for large number of tasks (on my laptop it fails at around 1000 forked processes). Both of these combined appear to work well with functions expensive in terms or CPU work (e.g.for i in range(10**6)) and IO waiting time (e.g.time.sleep(1)).Why not use asyncio for concurrency instead of threading?I think asyncio is a good choice over multi-threading when we can modify a blocking function into an awaitable coroutine. If we want/must use a blocking function (e.g. we can't modify it into asyncio coroutine because it's from some library we can't modify) and we want to make it concurrent, asyncio providesloop.run_in_executorwhich relies on multi-threading anyway.