busbud/coding-challenge-d — pull request #2 by phiiil (2014-09-02 13:27:43+00:00)
Title: Parallel image processing with Python Multiprocessing

Used the python multiprocessing library to operate on images in parallel. A process pool is initialized with the number of CPUs available on the machine. Images are store in ./processed_images/. The Pillow library already makes use of multiple CPUs internally. This means that most operations will split RGB images in 3 bands and operate in parallel on 3 CPUs. The combination of both strategies should make full use of the available CPUs.

Currently the last 3 crops are processed in a serial manner. More work could be done to run those 3 "crop and save" operations in parallel. I ran into a limitation of the default multiprocessing Pool implementation where it's subprocesses cannot spawn child processes without extending the library. Directly using non-daemonic processes instead of a Pool could provide of potential solution.