Repository: gwastro/pycbc

Issue:
Emcee sampling is broken
I haven't used this sampler in awhile but its broken now.

I can use it before PR #1308 where I tried commit ``9aff59fe`` and it works.

Then I tried PR #1536 (ie. commit ``35d65a7``) and I get this error:
```
Traceback (most recent call last):
  File "/home/cbiwer/opt/pycbc-dev3/bin/pycbc_inference", line 4, in <module>
    __import__('pkg_resources').run_script('PyCBC===9aff59', 'pycbc_inference')
  File "/home/cbiwer/opt/pycbc-dev3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 738, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/cbiwer/opt/pycbc-dev3/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1499, in run_script
    exec(code, namespace, namespace)
  File "/home/cbiwer/opt/pycbc-dev3/lib/python2.7/site-packages/PyCBC-9aff59-py2.7.egg/EGG-INFO/scripts/pycbc_inference", line 359, in <module>
    sampler.run(end-start)
  File "/home/cbiwer/opt/pycbc-dev3/lib/python2.7/site-packages/PyCBC-9aff59-py2.7.egg/pycbc/inference/sampler_emcee.py", line 417, in run
    res = self._sampler.run_mcmc(pos, niterations, **kwargs)
  File "build/bdist.linux-x86_64/egg/emcee/sampler.py", line 172, in run_mcmc
  File "build/bdist.linux-x86_64/egg/emcee/ptsampler.py", line 261, in sample
ValueError: setting an array element with a sequence.
```

Same as master now.

Any idea what's going on here? It was tested?


Discussion:
I have an example on sugwg: ``/home/cbiwer/projects/inference_samples_group/test/run_pycbc_inference.sh``
@ahnitz Can you take a look?
@cdcapano @cmbiwer Yes, I'll take a look. 
@cmbiwer The emcee sampler still works, but I'm investigating the PT sampler. I can reproduce the error, so that's a start.
@cmbiwer @cdcapano. The PT sampler is not working because it is relying on setting a *class* variable *after* the pool is created. This is not possible with the multiprocessing module. Specifically, this has no effect. https://github.com/ligo-cbc/pycbc/blob/master/pycbc/inference/sampler_emcee.py#L286

Resolving patch:
diff --git a/pycbc/inference/sampler_emcee.py b/pycbc/inference/sampler_emcee.py
--- a/pycbc/inference/sampler_emcee.py
+++ b/pycbc/inference/sampler_emcee.py
@@ -250,6 +250,17 @@ def write_results(self, fp, start_iteration=0, end_iteration=None,
                                     max_iterations=max_iterations)
         self.write_acceptance_fraction(fp)
 
+# This is needed for two reason
+# 1) pools freeze state when created and so classes *cannot be updated*
+# 2) methods cannot be pickled. 
+class _callable(object):
+    """ Create a callable function from an instance and method name"""
+    def __init__(self, instance, method_name):
+        self.instance = instance
+        self.method_name = method_name
+
+    def __call__(self, *args, **kwds):
+        return getattr(self.instance, self.method_name)(*args, **kwds)
 
 class EmceePTSampler(BaseMCMCSampler):
     """This class is used to construct a parallel-tempered MCMC sampler from
@@ -283,11 +294,9 @@ def __init__(self, likelihood_evaluator, ntemps, nwalkers, pool=None,
 
         # construct the sampler: PTSampler needs the likelihood and prior
         # functions separately
-        likelihood_evaluator.set_callfunc('loglikelihood')
-
         ndim = len(likelihood_evaluator.waveform_generator.variable_args)
         sampler = emcee.PTSampler(ntemps, nwalkers, ndim,
-                                  likelihood_evaluator,
+                                  _callable(likelihood_evaluator, 'loglikelihood'),
                                   likelihood_evaluator._prior,
                                   pool=pool)
         # initialize


Test patch: