kcarnold/joblib — comment on commit ba6dcb3609975e35a78657fdc5a6308baff3b0de
File:  line 0
By GaelVaroquaux:

Hi @kcarnold , thanks a lot for your contribution. It is great to have someone watching my back and fixing my errors.

This specific commit means that all the numpy sublasses will be saved using the slow, pickle-based mechanism. This will mean a strong performance hit for things like memmap or matrices.

It seems to me that it should be possible to pickle everything but the data '.data' attribute of the numpy array, and route the data to an npy. Out of the top of my head, I do not know how to do this, but I think that it would be an interesting alley to explore. I have create a branch on my github 'fix_pickling' that is currently simply a copy of your branch, and in which I am planning to explore such options.