Change the ModuleCache to only take the lock for actual compilation.

Also, once we acquire the lock do a refresh of the cache and try to
pick up modules compiled by others.
I think a clean up is only useful at the first import of the cache and when
we exit. I would also skip the clean up the other time when we start a
function compilation.

On Sat, Dec 13, 2014 at 2:25 PM, abergeron <notifications@github.com> wrote:

> In theano/gof/cmodule.py
> <https://github.com/Theano/Theano/pull/2321#discussion-diff-21790574>:
>
> > @@ -617,7 +617,18 @@ def __init__(self, dirname, check_for_broken_eq=True, do_refresh=True):
> >      Older modules will be deleted in ``clear_old``.
> >      """
> >
> > -    def refresh(self, age_thresh_use=None, delete_if_problem=False):
> > +    def _get_module(self, name):
> > +        if name not in self.module_from_name:
> > +            _logger.debug('loading name %s', name)
> > +            self.module_from_name[name] = dlimport(name)
> > +            self.stats[1] += 1
> > +        else:
> > +            _logger.debug('returning compiled module from cache %s', name)
> > +            self.stats[0] += 1
> > +        return self.module_from_name[name]
> > +
> > +    def refresh(self, age_thresh_use=None, delete_if_problem=False,
>
> That would be possible, but I would rather skip the cleanup entirely when
> doing the refresh at each node compiled. There is a full refresh just
> before each function compilation anyway.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/Theano/Theano/pull/2321/files#r21790574>.
>