scikit-learn/scikit-learn — comment on commit 7769275dc4fb9f5f8ea6448ecfbaffe4fdcbf1a6
File:  line 0
By mblondel:

libsvm uses an LRU cache so it shouldn't pre-allocate the memory.

I think that setting the cache size to a too small values goes against our principle of sane default parameters. Even as a scikit-learn developer, I'll probably forget to reset the parameter to a sane default value every time I use it. As long as the system doesn't crash, i.e., as long as a `MemoryError` exception is properly raised, I don't see the problem with setting the cache size to 500 MB. We should write a unit test for that but it may not be easy.

Regarding grid-search, looking at the grid search source code, yes there's gonna be a problem because it keeps all the estimators in memory. We need a version of grid search that can release the estimators so they get garbage-collected.

In the particular case of SVC/SVR, IIRC, Fabian has wrapped the cross-validation functionality built-in libsvm so if you use that, I'd guess that the kernel cache is reused across folds (but I haven't checked).