joblib/joblib — pull request #80 by ogrisel (2013-09-02 12:35:09+00:00)
Title: [MRG] [Optim] Do not compute hash when dispatching with Parallel

The MemmapingPool's custom reducer for large numpy arrays hash them to generate unique temporary filenames for memmaping. However when called through the public `joblib.parallel.Parallel` API, we are guaranteed that the temp folder is dedicated to a single call, hence hashing the content of the array is not necessary.

The second optim is to pre-warm newly dumped arrays to load them in the OS disk cache. This can be useful on slow spinning disks that would be penalized by concurrent random access to sub slices of the array concurrently in child processes.