jaberg/hyperopt — Issue #89: partial_refresh
Author: jaberg
Created: 2012-02-24 19:23:04+00:00
State: open

@yamins81, one consequence of putting so many things into the results dictionary is that now refresh is pretty slow. Even within rowland it takes 5 seconds with about 300 trials. This makes some sense because the trials document list pickles to about 35 MB. If we're aiming to scale up, we're going to need something like a "partial_refresh".

What do you think of adding the sub-document retrieval feature of mongo, so that

`trials.partial_refresh({'spec': 1, 'result': {'loss': 1}})`

would update just the spec subtree and the result.loss subtree (which happens to be just a scalar).